Concourse

Inline task definition

You shouldn’t make a habit of doing this, but here’s a link to a script that’ll inline task definitions, for the rare case when you want a one-off task definition.

Concourse on Linode

Some notes on running Concourse from a linode box:

  • You can run the web command and the worker command on the same machine. The web machine can be on a 1GB ram linode, it doesn’t take that much resources.
  • While doable on the 1GB ram plan, you should really run the workers on at least the 2GB ram plans. This is more for storage than anything else.
  • Using a linode is a better plan long term over getting a NUC so long as you stay under the 16 GB plan. Depending on your usage, the other benefits (not having to care about hardware issues) might even extend this to that.

As with the other services I maintain, the setup is managed inside of an ansible playbook.

Issues

I discovered the hard way that using the 1GB “nanode” plan was not a good plan. The disk very quickly filled up, in addition to everything being slow as molasses. Once I migrated the machine to the 2GB plan, I ran into issues with the volume space not being resized (concourse creates a worker volume logical volume with $TOTAL_DISK_SPACE - 10GB of space), then further issues with the system thinking that a volumes which were deleted in fact weren’t, etc.

Worker.beacon.forward-conn.failed-to-dial

See this issue

Remove $CONCOURSE_WORK_DIR/garden-properties.json before each time a worker starts.

Resizing the Worker Volume

See this issue.

# On a machine with fly
fly -t $TARGET land-worker -w $WORKER_NAME

# On the worker
sudo systemctl stop concourse_worker

# Back to fly
fly -t $TARGET prune-worker -w $WORKER_NAME

# Back to the worker
sudo umount -f /opt/concourse/work_dir/volumes
sudo sync
sudo losetup -d /dev/loop0
sudo rm -rf /opt/concourse/work_dir/volumes.img
sudo reboot

Pruning the worker (which really only needs to happen before the reboot) tells concourse to ignore any volumes that may or may not exist. Invoking land-worker may or may not actually do things.

Darwin Worker

I wrote something on this a few years back. Which is, of course, out of date (at least, in regard to houdini).

Here’s my current launchagent (~/Library/LaunchAgents/com.rachelbrindle.concourse.worker.plist):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>AbandonProcessGroup</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>Label</key>
    <string>com.rachelbrindle.concourse.worker</string>
    <key>Nice</key>
    <integer>0</integer>
    <key>ProgramArguments</key>
    <array>
    <string>/Users/you/concourse/worker.sh</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/usr/local/var/log/concourse_darwin_worker.log</string>
</dict>
</plist>

And the corresponding worker.sh:

#!/bin/sh -l

cd /Users/you/concourse
/usr/local/bin/concourse worker \
    --work-dir /Users/you/concourse/darwin_work_dir \
    --tsa-host $CONCOURSE_HOST:2222 \
    --tsa-public-key /Users/you/concourse/keys/web/tsa_host_key.pub \
    --tsa-worker-private-key /Users/you/concourse/keys/worker/worker_key