Building a Vim-like XMonad – Prompt, Task Groups, Topical Workspaces, Float Styles and More

Update: I’ve formally made a package called VIMonad with all the features discussed here (as well as other more interesting ones) here.

About a year ago, a friend of mine reading Computer Science introduced me to XMonad. At that time the window manager I was using - or rather stuck with - is Aqua from Mac OS X, and I was fanatically resorting to tmux as a temporary replacement for all purposes. This might be one of the most important moments for my working environment, since from that time I was enchanted by the simplicity of a Linux system and switched to Arch Linux + XMonad permanently.

Now one year later I’ve had enough experience (I can’t say I’m a Haskell master though) and it’s a good time to review my customizations and share my tips.

As the title indicates, my starting point is with Vim in mind. For me Vim is such an enlightening piece of software that it does not only count as one of the best editors, but also offers a lot of metaphors on improving efficiency, especially for a keyboard-driven workflow.

What I miss from the start

  1. Mod-o and Mod-i to navigate through the window history

    jump to discussion

  2. better splitter and buffer (window) integration; allow each split window to show multiple buffers (windows)

    jump to discussion

  3. improved command line control through XMonad.Prompt; especially integrate frequently-used commands into the prompt to make a dynamic system like Alfred

    jump to discussion

  4. topical workspaces with terminals starting inside with custom directories (there are implementations for this in Contrib but I don’t think the approach is elegant enough)

    jump to discussion

  5. dynamic renaming, reorganization, removement of workspaces; but at the same time each workspace should still be linked to a shortcut i.e. Mod-1 to Mod-9, Mod-- and Mod-=

    jump to discussion

  6. Mod-/ to search the windows; Mod-m and Mod-' to mark windows and jump to them

    jump to discussion

  7. manage windows in groups according to their different roles/tasks; I call them task groups. More specifically
    • Mod-] <specifier> and Mod-[ <specifier> cycles through the particular group
    • Mod-d <specifier> quits all windows in that group
    • Mod-S-] and Mod-S-[ switches between groups
    • Mod-S-o <specifier> and Mod-S-i <specifier> steps through the window history for that group

    Here <specifer> is a single or a sequence of keystrokes to represent a particular task group. As an example I have the following in my xmonad.hs

     b   --> Vimb
     v   --> Vim
     z   --> zathura
     m   --> mutt
     f   --> finch
     r   --> ranger
     t   --> idle xterms
     S-t --> all other xterms
    

    Note that as a convenience a special specifier is designed for each action. For example, Mod-] <specifier> and Mod-[ <specifier> have specifiers [ and ] that will cycle through the current group (the group of the currently focused window); similarly pressing Mod-d d will delete windows in the current group.

    jump to discussion

  8. expanding on the concept of task group in the previous point, each group has its particular float styles and pressing Mod-t and Mod-S-t shall cycle forward/backward through these styles. For example, I have defined for my ranger instances such that pressing M-t once will make it float and occupy the lower half of the screen; pressing it again makes it occupy the upper half of the screen; a third time sinks it back into the layout.

    jump to discussion

  9. able to toggle a few windows on and off (floating). Think of it as the dock in the Mac OS system; sometimes you need it but sometimes you don’t; and the best way to make it readily available without clustering the workspace is to have it hidden in the background and activated via a key. This is essentially XMonad.Util.Scratchpad from Contrib; however, the original scratchpad does not provide workspace-specific scratchpads, so all workspaces have to share the same scratchpad. My extension will work around this limitation and allow a different scratchpad in each workspace, toggled with the same key sequence.

    jump to discussion

  10. a wallpaper system that allows easy previewing and changing of the wallpaper. Press M-x to make all windows half-transparent to show the wallpaper (I call this gallery mode); press it again to go back to normal mode. Press M-S-x to switch to the next random wallpaper - if it’s in normal mode, turn on the gallery mode for a few seconds (to show the newly changed wallpaper) and go back to normal; if it’s already in gallery mode, do nothing more.

    jump to discussion

OK, enough said. I think I might have left out a few features but there is already too much to talk about.

Implementing the missing features

Window history navigation

As a starting point, XMonad.Group.Navigation provides a function nextMatch that can navigate back in history like this nextMatch History (return True) (shown as example in the doc). Here (return True) is the predicate to match all windows in history. However this is pretty useless since as soon as you go back to the last visited window the previous window becomes the new last visited one and essentially all this does is to toggle between two windows.

With a little thought though we can devise this simple barebone algorithm to improve upon that and allow for two-way navigation like in Vim

  • before we go backwards in history we mark the current window (in some storage) and go to the next unmarked window in the history. So now pressing M-o multiple times would skip windows previously navigated to since they’ve already been marked.
  • before we go forward in history we unmark the current window and go to the next marked window in the history. This requires a little more thought to bend the head around but think about this: when you go forward in history you are revisiting the windows that have previously been visited via M-o, so these windows must have been marked. At the same time you are annulling the effect of M-os so that’s the reason for unmarking.

The algorithm is fairly simple so the details of the implementation are left to you to work out.

Window groups

To have multiple buffers in the same splitter, or in other words, multiple windows occupying the same tile, all one needs is a tabbed layout embedded in each tile. Luckily this has already been implemented by other people. Check out the tiled tab groups in XMonad.Layout.Groups.Examples

Note: the original tiled tab groups seem to have a few bugs to prevent consistent redrawing of the tablines. If you encounter any such problems you can contact me for a fix

A preview of the effect:

An implication of this change is that now EVERY WINDOW becomes a tab. Additionally, new windows won’t change the outer layout in anyway - they just start as a new tab in the current group. This essentially eliminates any need for application-level window tabbing: now you don’t need suckless tabbed for tabbing your xterm windows, or Firefox/Chrome for their ostentatious but aesthetically un-unifiable tabs. The responsibility of window management truely falls to XMonad and it does it tons better (I actually hope that Vim can delegate window management to XMonad too but apparently many functionalities can’t be shared across in this way). Just a few reasons why this is so much better:

  • one single keystroke to move the current tab to another group, in an entirely visual way. Still remember the ‘groups’ from Firefox? You have to click the zoom button, drag the tab to the desired group and all. Also, now you can mix all sorts of ‘tabs’ together - you can put a xterm tab with a browser tab - anyway you want.
  • shifting tabs with ease - again, single strokes to move the tab left/right in the group
  • create new groups with ease - move any tab out of the current group to form a new group
  • unified look and keyboard shortcuts

This is precisely the reason why I’ve changed almost all my applications to those without window tabbing e.g. Vimb for browsing. This makes everything so much simpler - every window can be treated in the same manner as if it’s just a buffer in Vim.

All the glories about prompts

Prompts are an interesting addition to XMonad as they allow manual tasks to be performed anywhere and anytime in an unintrusive manner. However, I’m not entirely satisfied by the line of prompt systems included in Contrib - most of them only allow input, and have neglected another important feature of prompt - showing real-time output of the query. This can be best illustrated by Spotlight from Mac OS X or better yet, Alfred. Both apps attempt to show the user search results from the query and allow the user to easily go to any result. I feel that XMonad’s Prompt should do the same thing.

Here I’ll list a few prompt systems I’ve designed myself. Some are more complicated than others (might span a couple hundred of lines of code) and I won’t indulge into the details. To better understand how a prompt system should be written it’s better to consult the documentation directly at XMonad.Prompt. The main purpose here is to show what we can do rather than how it can be done.

Information system

The information system consists of a calculator (calc underneath) and a bunch of dictionaries (sdcv underneath). Press Mod-c <character> will activate the corresponding prompt i.e. Mod-c <digit> activates the calculator and puts that digit into the prompt whereas Mod-c <letter> activates the dictionaries and again puts that first letter into the prompt. In addition Mod-c <Return> will activate the prompt taking the words from the clipboard as the input.

When checking definitions for words one can press ` to switch between different dictionaries e.g. WordNet, Thesaurus, etc.

Pressing <Return> for the calculator will copy the result into the clipboard whereas for the dictionaries will pronounce the word using espeak.

A screenshot of the calculator:

And an example of using Chinese dictionaries:

Vimb prompt

Vimb stores history and bookmarks in plain-text and that makes building a prompt for it (think about Omnibox for Firefox) a breeze. My implementation just repeatedly greps the words on the prompt in Vimb’s bookmark and history file and outputs the result in formatted columns; and when the input is empty shows the last 10 visited URLs.

Taskwarrior prompt

I was once attracted by Taskwarrior and had since written a complete prompt system for it. Pressing <tab> and <S-tab> will autocomplete tasks and it also shows the real-time output for the filters used in the command. In addition, it autocompletes projects, due times, commands, etc. I’ve also integrated taskopen into the system such that pressing <Return> on any focused task automatically opens the notes file for it.

Topical workspaces

The implementations in Contrib on topical workspaces demand the user to put all the configurations into xmonad.hs. I found this too rigid and static - what if I just have created a new directory (a context) and want to start the workspace inside?

As a solution to my quagmire, I’ve conjured up an entirely different approach - managing topics by tags. Every tag is represented by a directory, and a file/directory with multiple tags will be hard-linked in all those respective directories.

An example tag structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
$ tree -d ./dev
./dev
├── algorithm
│   └── web_crawler
├── applescript
├── c
│   └── va_list
├── c++
├── graphics
│   └── opengl
├── java
│   └── swing
├── keyboard
├── keyword
│   └── static
├── latex
├── matlab
├── mmd
├── obj_c
│   ├── arc
│   │   └── weak
│   ├── block
│   └── category
├── os
│   ├── android
│   ├── blackberry
│   ├── ios
│   │   ├── coredata
│   │   └── scrollview
│   └── osx
│       └── uti
├── python
├── regex
├── unix
│   └── bash
│       ├── condition
│       ├── find
│       ├── grep
│       ├── osascript
│       ├── sort
│       ├── string
│       └── xargs
└── web
    ├── ip
    ├── js
    ├── php
    ├── xml_rpc
    └── zend

Then the task of switching to a particular topic is as simple as searching in the tag database for the tag, creating a new workspace, and storing the path of that tag directory for this workspace. After that load the path for any newly spawned xterms, etc.

The searching of tags, again, is achieved via a prompt.

Dynamic workspaces

There is already XMonad.Actions.DynamicWorkspaces but I feel like wanting a little more.

What I’ve added upon it:

Assign each workspace to a symbol

This is achieved via a customly defined symbol stream. It is similar to the Enum stream of Char in Haskell but slightly different - the order of the symbols is adjusted to maximize keyboard efficiency. For example, I’ve defined the symbol stream for my workspaces as follow

`123457890-=...

The ... refers to any other symbols e.g. obtained from enumFrom '='. You might notice that 6 is left out of the symbol stream and that’s right - I’ve specifically assigned Mod-6 to toggling between last visited workspaces, as a tribute to <Ctrl-6> in Vim. The first symbol, ` denotes the temporary workspace and cannot be removed.

Why is binding each workspace to a symbol useful? Because then one can perform many tasks with these symbols efficiently.

For example

  • Mod-<symbol> switches to the workspace with that symbol
  • Mod-S-<symbol> shifts the focused window to the workspace with that symbol
  • Mod-C-<symbol> swaps the current workspace with the workspace with that symbol

The symbol stream is also helpful for maintaining a consistent ordering of the workspaces in the dynamicLog.

Bind renaming/adding workspaces to the topic selection prompt I explained in the previous section

The original DynamicWorkspaces implementation simply allows you to type any string as the name of the current/new workspace. What I’ve done is to replace this prompt with my prompt for topic selection - so renaming/adding workspace would at the same time switch the workspace to the given tag/context (and create the new tag directory if necessary).

Window jumping

This is the easiest task of all. Simply use XMonad.Prompt.Window for Mod-/ window searching and XMonad.Actions.TagWindows for Mod-m and Mod-' window marking.

Task groups

The key to task groups is carefully modelled Query Bools (see manageHook).

Basically what I’ve done is defining the Query Bools that match the windows in each task group.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
windowGroups = siftedWindowGroups
    [ -- vimb instances
      def { filterKey = "b"
          , filterPredicate = className =? "Vimb"
          , onAbsence = vbPrompt
          }
      -- zathura instances
    , def { filterKey = "z"
          , filterPredicate = className =? "Zathura"
          , onAbsence = spawn "zathura"
          }
    ]

Using ifWindows from XMonad.Actions.WindowGo you can select the windows using the Query Bool; after that, you can pretty much imagine how to extend from it.

To ensure mutual exclusion among groups I’ve also made it such that the task group of a window will be the first matched group in the definition list i.e. so in the above example, a vimb instance will never be considered a zathura instance.

Some interesting Query Bools I’ve made:

  • isInCurrentWorkspace: matches windows in the current workspace
  • currentGroupQuery: returns the Query Bool of the task group for the currently focused window

What all remains is to use list comprehension to generate all the possible key-chords for the actions. So for the above example we will probably generate Mod-[ b Mod-] b Mod-d b Mod-[ z Mod-] z Mod-d z, etc.

Float styles

This should be relatively straightforward, given that the concept of task groups is already established in the previous section. Use currentGroupQuery to get the task group of the current window and cycle through the pre-defined styles (just plain ManageHooks) accordingly. Of course you’d need to store the index of the current style for each window so on next invocation of style-switching it will jump to the correct style, but it should be trivial to implement.

Per workspace scratchpads

The main technique for getting a per workspace scratchpad is through exploiting the original namedScratchpadAction function from XMonad.Util.NamedScratchpad like this

1
2
3
4
5
6
mkPerWSScratchpad cmd = do
    curr <- gets (W.currentTag . windowset)
    con <- perWSScratchpadContext curr
    dir <- getCurrentWorkspaceDirectory
    let csterm = UniqueTerm con cmd "" in
        namedScratchpadAction [ NS "cs" (uniqueTermFullCmd dir csterm) (isUniqueTerm csterm) idHook ] "cs"

So that each time the function is triggered the Query Bool associated for selecting the scratchpad changes according to the workspace. The code above might be a little abstruse to understand, but basically

  1. perWSScratchpadContext gets a particular context for the given workspace (a unique string for the workspace)
  2. UniqueTerm constructs a shell command that will spawn a xterm given a command to run and write the context obtained from the previous step into its own appName
  3. isUniqueTerm checks whether a window is the particular xterm window obtained before by checking its appName field

There are more subtle details to consider - for example, what happens when a workspace’s name changes? that will change the context obtained from 1 and subsequently lose the scratchpad associated with the old name. To solve this problem we have to introduce another level of indirection: create a map that associates each workspace with a unique handle, and use that handle to generate the context string. When the workspace’s name changes it should still be made to point at the same handle so the old scratchpad is still valid for use.

Dynamic wallpaper system

This last part is actually not that necessary in terms of functionality; but I still made it for the sake of my aesthetics. XMonad.Hooks.FadeInactive provides the function fadeIf to paint arbitrary transparencies to windows matching a Query Bool and if you’ve followed through this long post then you might immediately get the idea.

The described gallery mode is nothing more than painting transparencies to all windows; whereas the normal mode only excludes the focused window.

The trick for going into gallery mode for a few seconds and then going back is to use XMonad.Util.Timer, which provides a simple interface to construct a timer and do something when the time is up.

Of course, you still need to write your own script or install certain applications to actually switch the wallpaper, but that should be a piece of cake.

Conclusion

I have imagined that this would be a very long post and indeed it is. During writing I myself had to pause for a few times to try to remember what I was doing with that particular part of the code and what I was trying to achieve. To be honest I haven’t talked much about the implementation details in most cases. This post is more about summarizing my current understanding and imagery of a modern - or rather, geeky - window manager and what I have done to approach my ideals. Maybe in the future my ideals will change again and then I should redo/rewrite everything. But as always, joy lies in endless tinkering / 折腾就是快乐.

Nov 5th, 2013

Pinboard Synchronization With Vimb/Vimprobable Bookmarks

When it comes to workflow, I’m a screen-estate fascist who disapproves of any ‘extra’ GUI elements like buttons and menu-bars. That’s why I switched to Vimperator from conventional idioms like Safari and Firefox a long time ago; and that’s also the reason that I switched to Vimb/Vimprobable from Vimperator in the end - if all I need is Vim-like features from Vimperator, why bother having Firefox in the first place?

All is well except a few things. One of them is what I’m going to talk about today, bookmark synchronization with popular online services. With light-weight browsers such as Vimb, you can’t really expect the developer to write extensions for such things (well, how many developers does Vimb have? Mhh, just one) However, this simplicity is also the beauty of such light-weight programs - usually you can just achieve what you want by a little bit of tinkering.

To start with, the social bookmark service I’m using is Pinboard and they offer an easy-to-understand, comprehensive api here. On the other hand, Vimb is even more straightforward on the treatment of bookmarks - it simply stores them in plain-text under its config directory.

With knowledge above we can easily build a synchronizer from scratch. The problem of synchronization is really about two smaller sub-problems:

  1. When the local storage (Vimb bookmark file) changes, reflect the change on the cloud
  2. When the cloud storage (Pinboard) changes, translate the change to local

For point 1 we can keep a backup of the bookmark file and use diff on it with the current file each time before the update. This gives us the changes, if any, done to the bookmark file since the last update and all remains is just doing the same thing on the cloud.

For point 2 Pinboard has offered a convenient query method /posts/update, which essentially returns the timestamp of the latest change. By retrieving this timestamp before each update cycle we can be sure that we won’t miss any updates on the cloud.

Of course this leaves out details such as which should be performed before which, how we should avoid unnecessary pulling from the cloud given that some of the changes on the cloud are done by the script itself; but the main idea is still the same.

For completeness the whole script is here for download. The main abstract of the code is shown below

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
# $1 is the method to query upon, $2 is the file to direct the output, while all the subsequent arguments
# in the format 'name=value' are appended as data arguments to
# the query
#---------------------------------------------------------------
function pinboard_api() {
    local datastring= data=
    for data in "${@:3}"
    do
        # we need to escape the single quote..merr...
        datastring="$datastring --data-urlencode \$'${data//\'/\'}'"
    done
    eval $CURL_CMD$datastring $HTTPS_PREFIX$1$AUTH_TOKEN > $2
  local CURL_EXIT_CODE="$?"
  if [[ $CURL_EXIT_CODE != "0" ]] ; then
      error "Curl failed with exit code $CURL_EXIT_CODE"
  fi
}

#---------------------------------------------------------------
function verify_config() {
  if [[ ! -e "$PINBOARD_CFG" ]]; then
      error "Could not find $PINBOARD_CFG file! \nIt should contain two lines. Line 1 the username. Line 2 the API token. \nYou can create it manually, or use the \"login\" command."
  fi

  USER_NAME=`head -n1 $PINBOARD_CFG`
  API_TOKEN=`tail -n1 $PINBOARD_CFG`
  AUTH_TOKEN="?auth_token=$USER_NAME:$API_TOKEN"
}

# This will compare the current update with the last update and
# return true if there's new update
#---------------------------------------------------------------
function pinboard_has_update() {
  # See if we need to do an update at all
  echo "Retrieving current pinboard update time..."
  pinboard_api "/posts/update" "$CUR_UPDATE_FILE"
  if [[ ! -e "$LAST_UPDATE_FILE" ]]; then
      echo "$LAST_UPDATE_FILE does not exist..."
      return 0
  else
      # Compare current update with last update
      echo "Comparing last update with current update..."
      diff "$CUR_UPDATE_FILE" "$LAST_UPDATE_FILE"
      if [[ ! "$?" == "0" ]]; then 
          return 0
      fi ;
  fi
    return 1
}

#---------------------------------------------------------------
function synchronize_cmd() {
    local to_update_pinboard=false to_update_local=false
    echo "######## Checking changes on the server ########"
    verify_config
    pinboard_has_update && to_update_pinboard=true
    echo "######## Starting local-to-cloud synchronization ########"
    # first compare the two bookmarks file to see if there are any changes
    if [[ -e "$BOOKMARK_BACKUP_FILE" ]]; then
        echo "Detecting changes to the bookmark file..."
        local changes="`diff "$BOOKMARK_BACKUP_FILE" "$BOOKMARK_FILE"`"
        if [ -n "$changes" ]; then
            to_update_local=true
            # we need to grep for all deletions 
            echo "$changes" | grep '^<' | while read line; do
                delete_pinboard_bookmark_from_vimb_format "${line#< }"
            done
            # have to exit the script if error occurred
            (( $? != 0 )) && exit "$?"
            echo "$changes" | grep '^>' | while read line; do
                insert_pinboard_bookmark_from_vimb_format "${line#> }"
            done
            (( $? != 0 )) && exit "$?"
        else
            echo "No local-to-cloud synchronization required. Local bookmark storage is in sync with pinboard account."
        fi
    else
        to_update_local=true
        echo "Bookmark file has not been backed up before. Trying to insert all entries..."
        while read line; do
            insert_pinboard_bookmark_from_vimb_format "$line"
        done < "$BOOKMARK_FILE"
    fi

    if $to_update_local; then
        # we have updated the server so we should refresh the update time
        echo "Refreshing the last update time..."
        pinboard_api "/posts/update" "$CUR_UPDATE_FILE"
    fi

    echo "######## Starting cloud-to-local synchronization ########"
    if $to_update_pinboard; then
        echo "Retrieving bookmarks..."
        pinboard_api "/posts/all" "$CURL_TMP"
        # we need to convert the format into vimb's format
        xml sel -t -m '/posts/post' -c '.' -n "$CURL_TMP" | while read line; do
            # retrieve the attributes
            local href="`echo "$line" | xml sel -t -v '/post/@href'`"
            local desc="`echo "$line" | xml sel -t -v '/post/@description'`"
            local tag="`echo "$line" | xml sel -t -v '/post/@tag'`"
            # just adding the bookmark to tmp file
            insert_vimb_bookmark "$href" "$desc" "$tag"
        done
        # move the tmp file to overwrite the old bookmark file
        # this might be dangerous if there're unsynced changes in the bookmark file 
        # but for the time being we'd assume that the bookmark file is already in sync with
        # the cloud
        echo "Overwriting the old bookmark file..."
        mv -f "$BOOKMARK_TMP" "$BOOKMARK_FILE"
    else
        echo "No cloud-to-local synchronization required. $BOOKMARK_FILE is current."
    fi

    # back up the bookmark file
    if $to_update_local || $to_update_pinboard; then
        echo "Backing up the current bookmark file..."
        cp -f "$BOOKMARK_FILE" "$BOOKMARK_BACKUP_FILE"
    fi

    echo "Moving current update time to last update time..."
    mv -f $CUR_UPDATE_FILE $LAST_UPDATE_FILE
    echo "Done."
}

#---------------------------------------------------------------
function login_cmd() {
  if [[ ! -d "$PINBOARD_DIR" ]]; then
      mkdir "$PINBOARD_DIR"
  fi

  if [[ ! -d "$PINBOARD_DIR" ]]; then
      echo "$PINBOARD_DIR could not be made."
      exit 1
  fi

  echo "We will attempt to log into pinboard now."
  `which echo` -n "Please enter pinboard username: "
  local USER_NAME=`head -n 1`
  echo $USER_NAME > "$PINBOARD_CFG"

  $CURL_CMD -u $USER_NAME "$HTTPS_PREFIX/user/api_token" | tail -n 1 | sed 's/<[^>]*>//g' >> $PINBOARD_CFG
  local CURL_EXIT_CODE="$?"
  if [[ $CURL_EXIT_CODE != "0" ]]; then
      echo "Curl failed with exit code $CURL_EXIT_CODE"
      exit 1
  fi

  if [[ ! -e "$PINBOARD_CFG" ]]; then
      echo "$PINBOARD_CFG could not be made."
      exit 1
  else 
      chmod og-rw "$PINBOARD_CFG"
  fi

  echo "Created $PINBOARD_CFG."
  echo "Logged in!"
}

#***
# "Main"
#***

if [[ $# -lt 1 ]]; then
    error "Expected a command for the second argument. \nTry \"help\"."
fi

#***
# Process the second commands
#***
case "$1" in
    synchronize)
        [ "PINBOARD$PPID" = "$LOCKRUNPID" ] || exec lockrun -qna PINBOARD /tmp/lockrun.pinboard "$0" "$@" || exit 1
        synchronize_cmd
        ;;
    login)
        login_cmd
        ;;
    help)
        help_cmd
        ;;
    *)
        error "Unknown command $1. Try \"help\"." 2
esac

# exiting without any errors
exit 0
Nov 3rd, 2013

Implementing the Missing Drafting and Queueing Function in Octopress

It’s been a while since I first started using Octopress. Its simple and elegant way of writing has now become my de facto standard of writing, but there are still things to be desired. For one thing, the workflow consisting of writing, commiting, compiling and deploying is still not lazy enough; ideally the only thing that should be required of the writer is to write. Inspired by Dennis Wegner in his Synced and scheduled blogging with Octopress, I worked on a bash script to allow for friction-free, painless deployment process that goes totally automatic underhood.

A short summary of the tinkering

  1. Like in Dennis’es original post, two new folders called _drafts and _queue are added to the source directory to hold drafts and queued posts respectively.
  2. Again, just like in the original post, you can put any markdown files within the _drafts folder; as soon as there’s the published: true property within the YAML front matters like this

     ---
     published: true
     ---
    

    Then the draft will be moved to the _queue folder

  3. All the files within the _queue folder will be checked for its date property in the YAML front matters; if it’s before the current time, then the file will be moved to the _posts folder for publishing

So what’s added / improved?

  1. The script automatically checks for the validity of the YAML front matters during each move of any post; among other things, it will add the following properties if absent:
    • date: the last modified date of the file is used; this is useful if you don’t want to take the creation date for the draft as the date published on your website (which is the weird default behavior of Octopress) – in that case just leave this field blank and it will be added automatically when the draft gets moved to the queue (or when you put a file yourself into the queue)
    • layout: post is assumed as the default layout, which should be sensible enough
    • title: the file name of the post (subtracting the date prefix, if any) is used

    What all these mean is that the minimal setup you’d need to have for a post is

    Either

    1. touch a new file in _drafts with its file name as the title
    2. when finished, add published: true to the top (don’t forget the triple dashes)

    OR

    1. touch a new file anywhere with its file name as the title
    2. when finished, move it to _queue; the file will then be published during the next update

    The second approach eliminates the need to have the YAML front matters entirely

  2. The script checks for all changes within the octopress directory using git diff and git ls-files. This means compared to the approach in the original post, now the website will be rebuilt even when files like _config.yaml is changed

    Note that this might not be optimal if you’ve had _queue and _drafts tracked by git. In that case whenever a draft is changed or something gets moved into the queue then the source will be commited, changes pushed to the remote, website rebuilt and all. I myself have excluded these two directories from git because I don’t like the idea of having my drafts posting up to the cloud (I’m using GitHub as my host and I’m a little against having my unpolished thoughts shown in public) and I’ve had Dropbox versioning control these posts in the first place.

    However, if you’re pointing your source to a private remote, then it might be a good idea for versioning control your drafts. In that case I’d recommend you change the script to auto-commit the changes in _drafts and _queue but exclude website re-compiling in such cases by greping the output of git diff and git ls-files to see if all changes happen in these directories.

Where to download?

You can download the whole source file here. Below shows a preview of the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
case $1 in
    rake) 
        cd "$OCTOPRESS_HOME"
        rake "${@:2}"
        ;;
    update)
        # first move the files in the draft to queue if the draft property is set to true
        echo "######## Checking for finished drafts ########"
        # try to find all the markdown and md files
        find "$OCTOPRESS_DRAFTS_DIR" -type f \( -name '*.markdown' -o -name '*.md' \) | while read draft
        do
            # get the published property; if no published property is found then default to false
            if [ "`valueForProperty "$(YAMLFrontMatterOfPost "$draft" false)" 'published'`" = 'true' ]; then
                new_post_name="`fullNameOfPost "$draft"`"
                echo "$draft: Published is true, moving to queue folder (renaming to $new_post_name)..."
                mv -n "$draft" "$OCTOPRESS_QUEUE_DIR/$new_post_name"
            fi
        done
        # second move the files in the queue to the post directory if the date specified is in the past
        echo "######## Checking for publishable posts in the queue ########"
        find  "$OCTOPRESS_QUEUE_DIR" -type f \( -name '*.markdown' -o -name '*.md' \) | while read post
        do
            currDate="`date '+%s'`"

            # get the post name to ensure that date etc are properly initiated
            new_post_name="`fullNameOfPost "$post"`"
            dateString="`valueForProperty "$(YAMLFrontMatterOfPost "$post")" 'date'`"
            dateString="${dateString% *}"

            postDate="`date -d "$dateString" '+%s'`"

            if ((postDate < currDate)); then
                echo "$post: Post date is in the past, moving to post folder (renaming to $new_post_name)..."
                mv -n "$post" "$OCTOPRESS_POSTS_DIR/$new_post_name"
            fi
        done

        echo "######## Checking for changes in the source directory ########"
        cd "$OCTOPRESS_HOME"
        to_update=false
        # first check if there is any change in the source branch; if there's then commit and update
        # diff --quiet checks whether there's change in the tracked files; the second command checks whether there's untracked file
        # the draft and queue folders are UNTRACKED; so you can basically do anything you want for the drafts 
        if ! git --no-pager diff --exit-code || [ -n "$(git ls-files --others --exclude-standard)" ]; then
            echo "Changes to the octopress directory detected."
            to_update=true
        elif ! [ -e "$OCTOPRESS_STATUS_FILE" ]; then
            echo "No last update status found." 
            to_update=true
        elif [ "`cat "$OCTOPRESS_STATUS_FILE"`" = 1 ]; then
            echo "Last update failed."
            to_update=true
        fi

        echo "######## Beginning update ########"
        if $to_update; then
            echo "Commiting changes in the source branch..." 
            # the add commit phase don't really count towards update result as it can give back error code if there's nothing to commit
            git add -A 
            git commit -m "Source updated" 
            echo "Pushing changes to remote..." \
            && git push origin source \
            && echo "Generating and deploying the website..." \
            && rake gen_deploy \
            && echo 0 > "$OCTOPRESS_STATUS_FILE" \
            || echo 1 > "$OCTOPRESS_STATUS_FILE"
        else
            echo "Nothing to update."
        fi
        ;;
    *)
        usage
        ;;
esac
Nov 3rd, 2013

孤独的引力

独生子的意义再明显不过,那就是孤独,孤立,孤寂,与他人的隔绝——简言之,这是些独自运转的星球。村上从中发现的吸引力正是孤寂行星所特有的磁场:在全然的自我隔绝里有极大的黑暗,一个人若无法自外界获得光亮与慰藉,就只能强迫自己独力支撑。他以为听到来自另一个人的回应,其实不过是两个星球的喃喃自语,他们谈话,却与彼此不相干。那种声响看起来如此相似以致被误认为是声的交汇,实际上他们只能相互碰撞,却无法融合。他们是孤立的个体,而予对方以取暖幻象。 —-阿不壳

这是一篇关于村上春树《国境以南 太阳以西》书评中的一部分。作者把我想表达的意思都完美地表达了出来–年轻一代内心的孤独感以及人与人间那种虚幻的维系(bond)。当初读这本书时是一口气读完的。村上很好地发挥出了他的特长,通过一个冷静、风趣的视角,又一次抓住了一个个文艺小屌们的心。反观村上的这种写作风格,对于一个场景更多地倾向于对环境(scene)的详写而人物(actor)的略写,同时极少地使用上帝视角,故给人一种亦实亦幻感。

返回主题,今天突然回想起这段话源于一个风马牛不相及的事。因为要复习GRE所以急于找一个像样的学习场所–家里是绝对不行的,因为书桌离床只有半米不到的距离,这个小小的实在是太容易逾越了。 辗转一番混到市图书馆,昨天试水一下午,发现虽然空调差、网差、地方小但却成功地提高了我的效率。觉得也许人就是这样矛盾的一种生物。你需要自己的自由,但却也需要其他人在你身边构成一种无形的威慑力,否则就太容易入深渊。这种人与人之间微妙的需要与被需要的关系恰如开头所述的星球与星球间孤独的引力互相间喃喃低语,以为是声的交汇,实际上只能相互碰撞。不管是图书馆、咖啡厅,还是学校、工作场所,你也许会见到无数的人–有些人甚至颇有眼缘;然而当对方离开桌子,走出你视野的一瞬间,你却也淡然地无动于衷,并且毫不犹豫地在下一个需要记性的时候把这些记忆清空。所以–你看到的是什么呢?也许只是自己孤独的倒影吧–你既不认识也不理解坐在你对面的那个你称作同类的生物,只是通过自己的一些臆测在自己的经历和认识上构筑了一个虚拟的同类。虽然只有这短短一瞬–你清楚你需要这个同类,也清楚那个同类需要你。于是你们继续按着自己的既行轨道缓缓运行而去,并在这一瞬后擦肩而过。

Aug 18th, 2013

无法(无限?)刷新的人生

不知从什么时候开始对下列图标产生一种莫名的恐惧感

回想起来,最早对这类图标的认识来源于浏览器–对于大部分网页来说,refresh的意思更接近于reset,作用是快速地回到那个初始的、你所熟悉的状态。然而现在这类图标被广泛地使用于各类信息类应用:比如社交类的人人、微博、facebook,新闻类的flipboard、新浪,etc。在这些应用中,刷新意味着更新,而每次更新都意味着看到这样的画面

我本人是无法理解能不停刷人人、微博的人的大脑构造的–在我看来,不管我怎么快地刷新都会有新的消息,而我看的速度永远追不上它刷新的速度。在这个信息时代也许这就是成王败寇的关键–如果你能驾驭这个信息的潮流(ride the tide of information),你就能时刻站在时代的最前线,抓住最新的机会;而如果你跟我一样,是一个难以追上时代的人,你就会成为一个被时代所遗忘的old-timer;而且跟时代的差距很可能会越拉越大。

这种思想其实已经渗入社会的各个角落:在硅谷,get the sh*t done远比get everything perfect更重要;大多顾客现在更关注的是一个商品的functionality(does it work?),而不是这个商品的持久性(does it stand the test of time?)。由此可见这个时代所强调的是接受信息和变化的适应力,而不是十年磨一剑的那种钻研精神。适应力意味着取舍的能力–什么可以略过,什么应该粗看,什么必须细看;而这种能力正是理想主义者们(old-time perfectionist)所最缺的:在一个理想主义者心中,他所追求的美与善往往是没有任何瑕疵的,所以任何细节,无论大小粗细,皆为同等重要;这也是为什么他们也许会在最无关紧要的地方浪费大把时间。

时代确实是变了。我想起多年前读的一篇讨论工匠者和他们的作品的文章,里面提到了人对于美的理解的变化–从前的人们欣赏的是一样工艺品的经久性(durability)和那份可以流传亘古的永恒之美(perennial beauty):其中的代表如石刻、木刻艺术;也现在更强调的是个性(individualism)、活力(power)、叛逆性(rebellion),如各类奇形怪样的现代艺术。文中提到了一个非常有趣的小故事(anecdenote):一个成功的木匠在他徒弟问及其成功的关键时拉开他自己做的一套木质家具的抽屉让徒弟看抽屉的内侧。徒弟惊奇地发现抽屉的内侧被打磨得光滑发亮。他问师傅为什么这么做–毕竟,有多少用家具的人会把抽屉完全抽出来看内侧怎样?师傅不出所料地教导徒弟道:做我们这一行就是要对每一个细节下足功夫。现在想到这个故事却有种强烈的讽刺感:在木匠逐渐被机器和工厂取代的今天,一个木匠存活尚且不易,一个连抽屉内侧都必须打磨好的强迫症木匠会怎么样呢?也许这个木匠一觉醒来发现人们不再喜欢把家具一代代传下去,也因此不再需要一套“完美”的家具,于是他所执着的人生理念以及对艺术的理解就这样无情地被时代的浪潮吞噬了。搞IT的人也许会有更深的感悟–你花了好多年辛苦研发出的技术,也许会因为另一样新技术的出现一夜间变得一钱不值。所以最后的结论是:不要变成那个木匠

可是在这个道理人人都懂的现在,还是有人冒着饿死的危险成为了那个木匠。你问这个傻子是谁?他就是乔布斯。乔布斯在领导Mac开发团队开发历史上第一台Mac时要求在每台Mac的机箱内侧刻上自己团队成员的签名。而且当大部分PC厂家想尽办法节省成本提高竞争力的时候他是唯一一个要求机箱内侧必须够COOL够漂亮的人。当然从历史的角度看第一台Mac卖得并不好;不仅仅是第一台Mac,在那个苹果公司最困难的年代,之后的好几款被后来的评论家称之为“超越时代的产品”的苹果电脑都销量平平。但是从长远的角度看,乔布斯的这种理想和完美主义无疑是帮助苹果重拾商机并在当今IT界鹤立鸡群的一个关键。甚至可以说他的这种思想塑造了大半个苹果公司–现今的很多苹果骨干都是被乔布斯的这种气质吸引来苹果公司的:比如现任设计总监的Jonathan Ive,他在做hardware design的时候会在每个细节上咨询最权威的专家,采用最好的技术–甚至有的时候只是为了薄零点几厘米,或是只是为了更加美观。

从苹果的例子看也许完美主义并不是那么不可救药。相同的道理,不能追上每一个刷新也不是注定被淘汰(苹果每年出一款手机,几款电脑的节奏虽被很多人看低过,但从每年发布日的火爆看这样的节奏反而更加帮助了它商业上的成功)。也许这个时代虽然表面上肯定的是消费主义(consumerism)和实用性(practicality),但是人们心中却藏着一份对完美的热烈的渴望–哪怕他们深知凭己之力无法实现这份完美,也不由自主地靠近和崇拜这份完美。从这点上看Steve的销售哲学是成功的

To sell a product to a customer, you have to make the customer believe in its beauty.

对于很多人来说Steve是一个怪人。他深信禅(Zen)的作用,终日满身臭气(这在晚年有所改善);他常常纠结于细节,比如在做呈现的时候关于展品的位置都要反复实验。但是对于很多人来说他也不是那么陌生。因为虽然他本人可能难以被理解,他的工艺品却得到了广泛的认同甚至追捧。他骨子里是那个次世代的木匠,总是举着锥子雕磨自己作品的每个细节不肯放手。但是就是这样的被时代抛弃的怪人,却通过他的作品给我们带来了一些那些曾经的,来自于次世代的感动。

Aug 11th, 2013

标准之囚笼

高度发达的社会总是伴随高度发展的标准。如果你是一个蚂蚁,你要懂得如何和你的小伙伴们一起搬运食物;如果你是一只蜜蜂,你要懂得怎么跳8字舞;如果很不幸你是一个人类,你要懂得交通规则、为人处世、生存之道。。。事实上,你要懂得如此之多的规则以至于你从小开始就必须花十几年的时间来学习这些规则。对于社会来说,这些规则是必要的–它保证了我们所熟悉的这个环境能按照我们常规的思维方式慵懒地运行下去。但是这份慵懒的代价呢?过多的标准。

标准无外乎两种–正标准反标准。正标准规定什么该做,比如“饭前要洗手”;反标准则相反–什么不能做,比如“不能随地大小便”。从几率学来说,正标准更有局限性:在满足的条件下,遵守这个标准意味着放弃其他千万种可能的行为;反标准则宽松的多–你可以干任何事,但唯独不能干这件事。个人感觉人类社会还是反标准多一些。大部分社会其实都建立在一种保守体制上,通过人类的恐惧感来加深标准。最明显的例子就是法律,一旦犯法就会被惩罚–在过去的封建或集权政治里这也许就意味着极刑,而正是这种极刑的存在加深了普通人对于王权(也就是标准的象征)的敬畏。虽然不那么明显,但是文化里面的反标准同样鳞次栉比–东方人的羞耻文化(shame culture)就是最好的例子:人跟人的交往非常注意彼此间的距离感,很多事都是不能做的,而一旦做了就意味着强烈的羞辱感。纵观历史,这种现象也是可以理解的–正标准的存活率显然要比反标准低的多:就跟科学论证一样,推翻一个正标准只需要一个反例;而推翻一个反标准则需要证明没有这个标准的千万种可能性要优于有这个标准的(千万种-1)的可能性。

不管是正标准还是反标准,人的行为总是受到了限制。而这种限制最大的表现就在于创意和想象力的枯竭。看到这里“破坏型创造”(destructive creativity)的字眼也许会从你嘴里脱口而出。对,要想提高创造力就得有破坏这些规则的勇气。可是我觉得很多人还是小看了这些规则对于创造力的危害。从小到大,学习了的这么多标准其实反复加深了我们的一种潜意识;用两个字概括就是“要好”。再用两个字概括,就是“怕输”。这也就是我上一节中提到的,自古社会开始我们对于恶,羞耻,罪,刑罚的本能性逃避和恐惧。创新总会意味着更高几率的失败, 和周围人的格格不入,以及随时被社会否定的危险。相对而言,恪守陈规,归于平凡也许是大多数人选择的道路。说到底,我们自称是动物界的强者,其实(大部分人)不过是微渺的食草动物,时不时为身边的风吹草动担惊受怕。

有了勇气,提起能砍开标准之囚笼的大斧就行了么?不见得。即使摆脱了社会的标准,身边人的标准,也许你永远挣脱不开自己的标准。为什么好多艺术家一炮成名后反而难有令人惊艳的作品?因为他们的经验、品味,在给他们带来优势的同时,也成为了抑制创造力的慢性毒药。更可怕的是成名后他们自身对自己的期望。害怕做出低水准的东西–这份害怕,反而逼得他们做不出任何高水准的东西。殊不知在创意的天堂里,一万件失败的实验品里才能有一件成功之作。一个总是给自己设定极高标准的人是一个骄傲的人,也是一个无法超越标准的人。他最后也许能做出极高标准的标准内作品,但那终究是中庸之作,是机械制品。要进入创意的殿堂,一个人首先得扔掉自己对自己的那份高评价,以一个什么都不懂的孩童的心去看待这个世界。

Aug 11th, 2013

人性之熵

The lovely crew of Cowboy Bebop; Click to view more

Samurai的浮世绘风格是大受好评的

不得不说,渡边的作品都很懒。不管是名噪一时的Cowboy Bebop,还是后来略低调的Samurai Champloo,主人公们的懒散都是刻画的一个重点。同其他大多动漫每集二十多分钟充斥各种悬疑、惊险、或是fantasy的做法不同,渡边的镜头里常会有主角们静默发呆的镜头,仅仅通过他们的表情和背景音乐来传递里面的微妙情感。

有大批人表示渡边这种吞吞吐吐的风格很小儿科:“看了20集没懂说什么!”;“没有主线到底是要怎样!”;“每集一个路人角色累死观众么!”即便是一些渡边的死忠也表示对很多剧情不能理解–最明显的莫过于Samurai Champloo的结局:三个刚刚为互相卖命并结下深深羁绊的人就这样在十字路口挥一挥手分道扬镳。。。就这样?看惯民工漫里遍地打肿脸说煽情话的观众们表示这不科学了:最后没有爱情,也至少有个基情吧!而等完整个credit依然没看到其中任何人回头的人也只能自扇巴掌了–“渡边果然不是一般的COOL啊。”

开始我也觉得有点小心酸;就像一直呆久了的人总会希望对方对自己有所怀恋;但是听着收尾那大气的Hip Hop我觉得有点明白了。其实渡边一直就是一个enjoy描写旅途的人。在他的眼中生活没有一个开始,也没有一个完满的结尾,有的只是延绵不断的过程。一个真实的存在过的角色可以有属于他自己独特的过去、现在和未来,但却很难像大部分影视作品中描写的那样,能把他的经历浓缩成一部惊心动魄、前承后合、逻辑通顺的500分钟人生剪影。也许当每个人回忆自己的人生,都会像渡边描写的情节那样的琐碎 吧–每一集都是人生旅程中互相独立的一点小事,虽然没有500分钟的悬疑大剧来的刺激过瘾,但每每想起这些既没特意去记也没特意去忘的细节,总会忍不住淡淡一笑。

记得从前读过的哲学书上讨论过的一个问题:现在的我是不是过去的那个我?这个问题看似简单,但到现在也没人想透。也许真的像钱老先生在《围城》里说的

(方鸿渐)他想现在想到重逢唐晓芙的可能性,木然无动于中,真见了面,准也如此。缘故是一年前爱她的自己早死了,爱她,怕苏文纨,给鲍小姐诱惑这许多自己,一个个全死了。有几个死掉的自己埋葬在记忆里,立碑志墓,偶一凭吊,像对唐晓芙的一番情感。有几个自己,仿佛是路毙的,不去收拾,让它们烂掉化掉,给鸟兽吃掉——不过始终消灭不了,譬如向爱尔兰人买文凭的自己。

所以当问起人生是什么,我想人生是跟皮影戏差不多的东西。千千万万个相似却又不同的被称做’我‘的影子,在千千万万个过去、现在、将来的互不干涉的时空里演着没有开始,没有结局,只有过程的小故事。人也是自然界里的一份子,虽然是一种高级动物,但却也逃不离的运作–每份经历和回忆,总会从一开始的连贯通顺,变成最后的一锅浆糊。你也许不再记得十年前那个女孩的名字,但是你却记得午后她低头的那个侧脸;你也许不再记得当年的自己喜欢哪个偶像,听什么歌,看什么片子,但却记得当时某个瞬间的心跳;更有甚者,你不再记得关于此情此景的任何东西,却莫名地感到一丝淡淡的熟悉感。

当走到那个路口,我想我应该可以潇洒地对过去挥一挥手,阔步向前。

See you cowgirl, someday, somewhere!

Jul 31st, 2013

Lynnard’s Very First Nerdy Rant About Blogging

Finally opened my own blogspace. I’ve been dreaming about this day for a long, long time but never got time to set it up properly. For one thing, the common blogging scene is so crowded with web-interface-backed alternatives such as Wordpress - which at best is cumbersome and not hacker-minded. An easy dig around the net reveals to me the gem in the sea: Octopress.

What is Octopress?

  • Write your blogs in Markdown with any editor (Sweet, vim it is)
  • Include code like skateboarding in the sky
1
2
$ rake new_post["I like to write blog in CLI"]
$ rake preview
  • Version control your blog like any code in Github

Gotcha

As a `Pandoc` loyalist, Octopress's choice of using `rdiscount` as its default `Markdown` compiler without a doubt unnerves me. Therefore the first thing to do is to integrate Pandoc into Octopress. Fortunately a little [post](http://drz.ac/2013/01/03/blogging-with-math/) helped me.

Unfortunately the Pandoc solution is NOT working for me. For the time being I’ll be settling with Octopress’s own codeblock feature or its github code fencing filter.

Gonna be different?

One biggest problem with Octopress is its poor support for themes (or rather, the dearth of the themes?), which leads to the fact that almost every Octopress blog looks exactly the same. To be fair the default theme looks perfectly fine to my eyes; but after seeing it being used with only changes in background wallpaper across countless other blogs it’s out of luck for me. A purist needs something simple, functional, and yet a little different. That’s why I decided to settle on slash. In future I might tweak the theme myself to suit my needs but all is good for now.

What about working on the go?

This, however, is a big problem. The un*xy set of utilities sported by Octopress on CLI affords great flexibility and extensibility when working on a computer, but when you are left with only an iPhone or an Android then all becomes a giant PITA. I do wonder from time to time why no phone manufacturer has ever introduced a phone with a proper Terminal interface - which would definitely thrill up a huge crowd of nerds - but apparently nerds (read 屌丝) are just out of consideration for any manager.

I’m to have a Blackberry Q10 in October so ideally there can be some solution for this great phone with its great QWERTY keyboard. One thing I can think of is to have a server running 24hr that monitors my Dropbox directory and recompiles my website whenever a change is made; at the same time use some text editor to edit the posts on my phone; or better yet, just use a SSH client and …

Oh forget about it.

Jul 30th, 2013