Hunter Ratliff

Taming noisy random HDD spin-ups between Time Machine backups

2026-03-14T00:00:00-07:00

This article details my strategy for absolutely minimizing the amount of times I must hear my external hard disk drive (HDD), used for macOS Time Machine backups, noisily spin up to the outright bare minimum. It has been a ride…

Fancy yourself comfortably in bed, slowly drifting off to sleep, when suddenly you’re jolted awake by:

Your browser does not support the audio element.

bzzzzZTT...whhhrrrrrrrRRRRR-TICK-TICK-TICK-HHHRRRRRRRRRRRR-tick-RRRRR

Imagine just casually watching a YouTube video or trying to focus on work, and then suddenly, out of the blue, BZZZT-WHIRRRRRR-TICK-TICK-HRRRR…

Reducing this to the absolute minimum is the mission I set out to achieve and detail here.

Setting the stage

My home computing setup consists of a M4 MacBook Pro that largely lives lid-closed in clamshell mode docked to a CalDigit TS5+ dock, which is then connected to two monitors, speakers, and other accessories. It is very convenient being able to have at minimum just the single Thunderbolt 5 cable to plug/unplug to switch between using my Mac like a desktop versus as a laptop (though, I often do also connect the MagSafe cable for AC power). In general, it lives this “desktop life”, and I have adjusted some settings to have it behave more like a desktop, most importantly being to disable the machine from automatically going to sleep when on AC power, allowing me to run simulations or any other task overnight without fear of the system interrupting it.

Simultaneously, I also strongly believe in the 3-2-1 backup rule, that my data should consist of at least 3 copies spread over 2 different types of storage media with 1 copy offsite. For years I have been a happy customer of Backblaze for that offsite component, but my history with the additional local copy has been a struggle, until I moved to macOS, which has an excellent built-in utility exactly for this purpose: Time Machine. All you need is an extra external drive with a bunch of extra space, ideally at least twice the Mac’s storage capacity. I happened to find a pretty good deal on an (overkill) 14 TB Western Digital external hard drive (HDD), so that is what I have living connected to my dock at all times.

Little did I know at the time what a rabbit hole this specific series of decisions was about to lead me down…

The initial problem

What I didn’t know at the time was:

My HDD is really quite noisy when it spins up… Your browser does not support the audio element.
Time Machine’s default scheduling options weren’t ideal for me.
The system will regardless spin up the HDD randomly even outside of Time Machine backups.

This had me feeling quite frustrated. While Time Machine is an absolutely lovely utility that makes good data backup habits incredibly easy, including encrypting your backups by selecting APFS encryption when configuring the volume, its user-facing options are extremely minimal. Your options for backup frequency are hourly, daily, weekly, or manually. For me personally, I think daily is fine, hourly is probably overkill, but ideally I would be able to have something in between.

Furthermore, even if I set Time Machine to only back up daily, for as long as the HDD remains mounted to my Mac, the system and/or the dock will prod at it randomly, causing it to spin up, even if the “Put hard disks to sleep when possible” setting is set to “Always”. Sometimes it would go an hour or few without spinning up; other times it would spin up several times within an hour. It was becoming extremely grating, and I was really starting to regret my HDD purchase. “Had I just paid quite a bit more money for an SSD-based external drive for Time Machine it’d be silent and without any of these issues!” I lamented.

It was too late for that solution now though. While I had bought it at a very good deal, that HDD still wasn’t so cheap that just spending my way out of the problem was remotely palatable. Surely I could work through these issues and somehow develop a solution, right?

The solution as designed

Assumptions

I outline here the solution implemented in my environment with my configuration: A M4 MacBook Pro running macOS 26 Tahoe connected to a dock (CalDigit TS5+) connected to an APFS-encrypted external HDD (WD Elements, 14 TB), where the MacBook by default lives connected to the dock unless I happen to be taking it out with me somewhere. I imagine this solution will be more broadly applicable than to this specific combination of equipment and configuration, but since I never pinpointed exactly every single different reason why my HDD spun up at all the various circumstances it did, just developed a robust solution that prevented it all, it is possible some of these steps may not be fully necessary nor complete on your own system. That said, I do feel this approach is quite robust.

Objective

The HDD spin-ups should be absolutely minimized to the bare amount necessary. As a baseline, this means it should only spin up for Time Machine backups alone, though I consider additional spin-ups when I re-dock the MacBook (after being disconnected) or when I first wake the MacBook from sleep or restart it acceptable. The key here is that spin-ups should be consistent and predictable, not happening randomly at any hour and regardless of whether the MacBook is actively being used, locked and idle, or set to sleep. Furthermore, I want Time Machine to perform backups at a few set times of my choosing per day, but I would also like for these backups to be skipped if the MacBook is set to Sleep. The solution should be fully in software and automated; it should not require me to remember to unplug/replug some physical cable or to manually run some script or Shortcut.

Solution

The solution I ultimately fell upon is to keep the HDD used for Time Machine unmounted at all times, only allowing it to be automatically mounted immediately before a Time Machine backup and promptly automatically ejected afterward. The small TimeMachineEditor app is the perfect solution for the Time Machine scheduling part here; I specifically have settled on three scheduled backups per day at 11:00, 16:00, and 21:00. I then use two custom-made LaunchDaemons to schedule the automation of mounting the HDD immediately before a scheduled Time Machine backup and to then, only after the backup is complete, eject the HDD once more.

The LaunchDaemons essentially each just run a shell script. The mounting script is scheduled to run 2 minutes before each scheduled TimeMachine backup and is responsible for first ensuring that the MacBook is not set to be sleeping before proceeding and then unlocking and mounting the APFS-encrypted HDD used for Time Machine backups; it also creates a temporary file to signify it was responsible for mounting the HDD. The ejection script is scheduled to launch 1 minute after each scheduled Time Machine backup; it first checks for that temporary file (as to not errantly disconnect the HDD if I had manually mounted it myself) and, if present, proceeds to then enter a polling loop where it checks whether Time Machine is still running or is finished (every 20 seconds for 5 minutes followed by every 1 minute for 40 more minutes) and, once Time Machine is detected as finished, it then unmounts the HDD volume, deletes the temporary file, and terminates.

I also created a Shortcut that allows me to manually mount the HDD easily in the event I ever want to interact with it solely in software, rather than needing to do some clunky unplugging and replugging of cables. And, of course, when I’m done with it I can just eject it normally as done with any external media.

Next, I’ll step through the exact implementation for getting this all set up.

How to implement the solution

Step 1 — Identify APFS volume UUID

First, you’ll need to be able to consistently identify which attached volume is your drive used for Time Machine. Fortunately, each drive has a universally unique identifier (UUID) associated with it that is constant that you can fetch.

Run:

diskutil apfs list

Locate the APFS Volume named your Time Machine volume (e.g., TimeMachine on my system), then copy its Volume UUID (not the container UUID). For example, on my system:

You will use it shortly in Step 5 (and later in Step 12) as:

TM_APFS_UUID="YOUR-UUID-HERE"

Step 2 — Create required directories

sudo mkdir -p /usr/local/etc/tm-dock
sudo mkdir -p /usr/local/sbin

These directories will be where the schedule file (tiny text file listing out the desired backup times), disk passphrase file (for unlocking the APFS-encrypted HDD), and scripts for the LaunchDaemons (and Shortcut) will live. These are made in the next few steps.

Step 3 — Create schedule file

Create:

sudo nano /usr/local/etc/tm-dock/times.conf

Example contents:

# One backup time per line (24-hour HH:MM)
00
00
00

Save and exit.

This is your single source of truth for scheduling for the LaunchDaemon side of things.

Step 4 — Store disk passphrase, ensure root-only access

Create:

sudo nano /usr/local/etc/tm-dock/passphrase

Paste your Time Machine disk encryption password (one line only). You set this password when you first configured the volume with APFS encryption. Most likely you had selected to have it saved to the Mac when you created it. If so, it can be found in the Keychain Access app (search for it in Spotlight; yes, the password lives here and not in the separate Passwords app); searching for the name of your volume (e.g., TimeMachine for me) in it is probably the fastest way to find it.

Secure the passphrase file:

sudo chown root:wheel /usr/local/etc/tm-dock/passphrase
sudo chmod 600 /usr/local/etc/tm-dock/passphrase

Security note: Only root can read this file. If someone has root access to your computer, disk encryption is not your limiting defense anyway and you have far bigger worries.

Step 5 — Prepare the mount script

Create:

sudo nano /usr/local/sbin/tm-dock-mount.sh

Paste:

#!/bin/zsh
set -euo pipefail

# ===== USER SETTINGS =====
TM_APFS_UUID="PUT_YOUR_APFS_VOLUME_UUID_HERE"
VOL_PATH="/Volumes/TimeMachine"
PASSFILE="/usr/local/etc/tm-dock/passphrase"
SCHEDULE_FILE="/usr/local/etc/tm-dock/times.conf"
# =========================

# Exit only if the most recent system wake was positively identified as a
# background DarkWake (PowerNap, maintenance, scheduled job).
# Any other outcome — full wake, unreadable log, unexpected format — proceeds,
# since a missed backup is worse than an unnecessary HDD spin-up.
# Parse the event-type label (field 4) from the pmset log:
#   "Wake"      = full user wake -> proceed
#   "DarkWake"  = background/PowerNap wake -> exit early
#   anything else / empty -> proceed (fail safe)
# The grep pre-filters to lines whose label is Wake or DarkWake followed by
# the same word in the description, excluding noise like "Wake Requests".
mkdir -p /tmp/tm-dock

last_wake_line=$(pmset -g log 2>/dev/null | LC_ALL=C grep -E '^\S+ \S+ \S+ (Wake|DarkWake)\s+\t?(Wake|DarkWake)' | tail -1)
wake_label=$(echo "$last_wake_line" | LC_ALL=C awk '{print $4}')

echo "$(date): wake_label='${wake_label}' last_wake_line='${last_wake_line}'" >> /tmp/tm-dock/idle-debug.log

if [[ -n "$last_wake_line" && "$wake_label" == "DarkWake" ]]; then
  echo "$(date): exiting early (DarkWake detected)" >> /tmp/tm-dock/idle-debug.log
  exit 0
else
  echo "$(date): proceeding with script (full Wake or no DarkWake detected)" >> /tmp/tm-dock/idle-debug.log
fi

# This job is scheduled at (scheduled_time - 2 minutes).
# Guard: only act if within +/- 3 minutes of any expected mount time
WINDOW_MINUTES=3

STATE_DIR="/tmp/tm-dock"
FLAG_FILE="$STATE_DIR/mounted_by_script"
mkdir -p "$STATE_DIR"

# Convert current time to minutes since midnight
now_min=$((10#$(date +%H)*60 + 10#$(date +%M)))

# Parse times.conf -> minutes from midnight
# Each valid HH:MM entry is converted to minutes since midnight and collected
# into sched_mins. Lines beginning with # or blank lines are skipped.
sched_mins=()
while IFS= read -r line; do
  [[ "$line" =~ ^[[:space:]]*# ]] && continue
  [[ "$line" =~ ^[[:space:]]*$ ]] && continue
  if [[ "$line" =~ ^([0-9]{1,2}):([0-9]{2})[[:space:]]*$ ]]; then
    h=${match[1]}; m=${match[2]}
    sched_mins+=($((10#$h*60 + 10#$m)))
  fi
done < "$SCHEDULE_FILE"

# This script is scheduled 2 minutes before each backup time. Check that the
# current time falls within WINDOW_MINUTES of the expected mount time for at
# least one scheduled backup. The +1440 % 1440 arithmetic handles midnight
# wraparound. If d > 720 we take the shorter arc around the clock face.
in_window=0
for s in "${sched_mins[@]}"; do
  mount_min=$(((s - 2 + 1440) % 1440))
  d=$(( (now_min - mount_min + 1440) % 1440 ))
  (( d > 720 )) && d=$((1440 - d))
  if (( d <= WINDOW_MINUTES )); then
    in_window=1
    break
  fi
done
(( in_window == 1 )) || exit 0

# If already mounted, do nothing (and don't set flag).
[[ -d "$VOL_PATH" ]] && exit 0

# Unlock (if locked). If already unlocked, this prints an error; we ignore it.
# Note: unlocking may mount automatically, but not always.
/usr/sbin/diskutil apfs unlockVolume "$TM_APFS_UUID" -stdinpassphrase < "$PASSFILE" >/dev/null 2>&1 || true

# If still not mounted, determine the current disk identifier (e.g. disk7s2) and mount it.
if [[ ! -d "$VOL_PATH" ]]; then
  DISK_ID="$(/usr/sbin/diskutil apfs list | /usr/bin/awk -v uuid="$TM_APFS_UUID" '
    $0 ~ uuid { getline; if ($1 == "APFS" && $2 == "Volume" && $3 == "Disk" && $4 ~ /^\(Role\):$/) { print $5; exit }
             else if ($1 == "APFS" && $2 == "Volume" && $3 == "Disk") { print $4; exit } }
  ')"

  # Fallback parsing (more robust): look for "APFS Volume Disk (Role): diskXsY"
  if [[ -z "${DISK_ID:-}" ]]; then
    DISK_ID="$(/usr/sbin/diskutil apfs list | /usr/bin/awk -v uuid="$TM_APFS_UUID" '
      $0 ~ uuid { found=1 }
      found && /APFS Volume Disk/ { for (i=1;i<=NF;i++) if ($i ~ /^disk[0-9]+s[0-9]+$/) { print $i; exit } }
    ')"
  fi

  if [[ -n "${DISK_ID:-}" ]]; then
    /usr/sbin/diskutil mount "$DISK_ID" >/dev/null 2>&1 || true
  fi
fi

# Confirm mounted and set flag (so unmount job knows it’s safe to unmount)
if [[ -d "$VOL_PATH" ]]; then
  echo "1" > "$FLAG_FILE"
fi

Replace the UUID line with your UUID from Step 1.

Make executable:

sudo chmod 755 /usr/local/sbin/tm-dock-mount.sh

In the script, you can delete the lines echoing to /tmp/tm-dock/idle-debug.log if you’d like, but I was using this log file a ton when troubleshooting getting this active/locked/sleeping gate to behave just right, only skipping backups when the Mac was set to sleep (but allowing them to proceed if the system was only locked or in active use). It is still a nice sanity check to confirm the scripts are behaving as desired.

To add, this was a huge rabbit hole all on its own. There were many methods I tried to use for querying the current system state, but none were consistent and many were difficult to exactly troubleshoot since the LaunchDaemons are executing in the system environment rather than the user environment. I ultimately found just querying the most recent Wake event from the pmset log to see if it was a DarkWake or normal (Full)Wake was the best way to figure this out.

Also, as stated early on, I do also have the “Prevent automatic sleeping on power adapter when the display is off” Battery setting enabled, meaning only locking my MacBook while docked should leave it in a full wake state even once the displays turn off automatically to save energy; only manually setting it to Sleep (which I do nightly or when leaving the MacBook for a while) should put it in a state where DarkWake events are occurring when these LaunchDaemons run.

Step 6 — Prepare the monitor + unmount/eject script

Create:

sudo nano /usr/local/sbin/tm-dock-monitor-unmount.sh

Paste:

#!/bin/zsh
set -euo pipefail

VOL_PATH="/Volumes/TimeMachine"
SCHEDULE_FILE="/usr/local/etc/tm-dock/times.conf"

STATE_DIR="/tmp/tm-dock"
FLAG_FILE="$STATE_DIR/mounted_by_script"
mkdir -p "$STATE_DIR"

# Only unmount if we mounted it
[[ -f "$FLAG_FILE" ]] || exit 0

# Tight polling: fast for first 5 minutes, then slower
FAST_POLL_SECONDS=20
FAST_POLL_DURATION_SECONDS=300   # 5 minutes
SLOW_POLL_SECONDS=60
MAX_MONITOR_SECONDS=2700   # 45 minutes

# This script is scheduled 1 minute after each backup time and monitors for up to 
# 45 minutes. These offsets define that window relative to each scheduled backup time.
WINDOW_START_OFFSET=1
WINDOW_END_OFFSET=45

# Convert current time to minutes since midnight
now_min=$((10#$(date +%H)*60 + 10#$(date +%M)))

# Parse times.conf -> minutes from midnight, same as mount script.
sched_mins=()
while IFS= read -r line; do
  [[ "$line" =~ ^[[:space:]]*# ]] && continue
  [[ "$line" =~ ^[[:space:]]*$ ]] && continue
  if [[ "$line" =~ ^([0-9]{1,2}):([0-9]{2})[[:space:]]*$ ]]; then
    h=${match[1]}; m=${match[2]}
    sched_mins+=($((10#$h*60 + 10#$m)))
  fi
done < "$SCHEDULE_FILE"

# Check whether the current time falls within [backup_time + 1, backup_time + 45]
# for any scheduled backup. The if/else handles midnight wraparound: when start > end 
# the window crosses midnight, so membership is tested with OR rather than AND.
in_window=0
for s in "${sched_mins[@]}"; do
  start=$(((s + WINDOW_START_OFFSET) % 1440))
  end=$(((s + WINDOW_END_OFFSET) % 1440))
  if (( start <= end )); then
    (( now_min >= start && now_min <= end )) && in_window=1
  else
    (( now_min >= start || now_min <= end )) && in_window=1
  fi
done
(( in_window == 1 )) || exit 0

# Only proceed if the mount script set the flag, and the volume is still mounted.
# If the volume has already disappeared on its own, clean up the flag and exit.
[[ -f "$FLAG_FILE" ]] || exit 0
[[ -d "$VOL_PATH" ]] || { rm -f "$FLAG_FILE"; exit 0; }

# Returns true if Time Machine is actively running a backup.
is_tm_running() {
  /usr/bin/tmutil status 2>/dev/null | grep "Running = 1" >/dev/null
}

# Attempts to unmount the volume up to 12 times with 2-second pauses between
# tries, to handle transient busy states (e.g. TM finishing a final write).
unmount_with_retries() {
  for i in {1..12}; do
    if /usr/sbin/diskutil unmount "$VOL_PATH" >/dev/null 2>&1; then
      return 0
    fi
    sleep 2
  done
  return 1
}

begin_epoch=$(date +%s)

# Poll until Time Machine finishes, then unmount. Uses a fast poll interval
# for the first FAST_POLL_DURATION_SECONDS, then switches to a slower interval
# to reduce unnecessary CPU wake-ups during long backups. Exits unconditionally
# after MAX_MONITOR_SECONDS regardless of backup state, to avoid the script
# running indefinitely if something goes wrong.
while true; do
  elapsed=$(( $(date +%s) - begin_epoch ))
  (( elapsed <= MAX_MONITOR_SECONDS )) || exit 0

  if is_tm_running; then
    if (( elapsed < FAST_POLL_DURATION_SECONDS )); then
      sleep "$FAST_POLL_SECONDS"
    else
      sleep "$SLOW_POLL_SECONDS"
    fi
  else
    unmount_with_retries || true
    rm -f "$FLAG_FILE"
    exit 0
  fi
done

Make executable:

sudo chmod 755 /usr/local/sbin/tm-dock-monitor-unmount.sh

Note that the temporary “flag” file "/tmp/tm-dock/mounted_by_script" is the important piece of insurance here that prevents this script from ejecting the HDD unless the HDD was mounted automatically by tm-dock-mount.sh in the first place (which creates the flag file).

Step 7 — Prepare the script that builds the LaunchDaemons

Create:

sudo nano /usr/local/sbin/tm-dock-generate-plists.sh

Paste:

#!/bin/zsh
set -euo pipefail

# Reads backup times from times.conf and generates two LaunchDaemon plist
# files: one to mount the Time Machine volume before each backup, and one
# to monitor and unmount it afterward. Run this script with sudo whenever
# times.conf is changed, then reload the daemons manually.

# Source of truth for backup schedule. Each line should be HH:MM format.
SCHEDULE_FILE="/usr/local/etc/tm-dock/times.conf"

# Scripts invoked by the generated LaunchDaemons.
MOUNT_SCRIPT="/usr/local/sbin/tm-dock-mount.sh"
UNMOUNT_SCRIPT="/usr/local/sbin/tm-dock-monitor-unmount.sh"

# LaunchDaemons must live here to be loaded by launchd at boot.
PLIST_DIR="/Library/LaunchDaemons"
MOUNT_PLIST="$PLIST_DIR/com.tmdock.mount.plist"
UNMOUNT_PLIST="$PLIST_DIR/com.tmdock.unmount.plist"

mkdir -p "$PLIST_DIR"

# Parse times.conf into a list of scheduled backup times in minutes since midnight. 
# Comments and blank lines are skipped. Hour and minute values are range-validated.
mins=()
while IFS= read -r line; do
  [[ "$line" =~ ^[[:space:]]*# ]] && continue
  [[ "$line" =~ ^[[:space:]]*$ ]] && continue
  if [[ "$line" =~ ^([0-9]{1,2}):([0-9]{2})[[:space:]]*$ ]]; then
    h=${match[1]}; m=${match[2]}
    (( h >= 0 && h <= 23 )) || continue
    (( m >= 0 && m <= 59 )) || continue
    mins+=($((10#$h*60 + 10#$m)))
  fi
done < "$SCHEDULE_FILE"

# Sort and deduplicate parsed times (@o = sort, @u = unique in zsh).
# Abort if no valid times were found.
mins=("${(@ou)mins}")
if (( ${#mins[@]} == 0 )); then
  echo "No valid times in $SCHEDULE_FILE"
  exit 1
fi

# Converts a time in minutes since midnight to a plist <dict> entry suitable
# for use in a StartCalendarInterval array.
hm_dict() {
  local mm=$1
  local hh=$(( mm / 60 ))
  local mi=$(( mm % 60 ))
  echo "      <dict><key>Hour</key><integer>${hh}</integer><key>Minute</key><integer>${mi}</integer></dict>"
}

# For each scheduled backup time, derive the mount and unmount fire times:
#   mount:   2 minutes before the backup (ensures HDD is ready in time)
#   unmount: 1 minute after the backup (gives TM time to start before monitoring begins)
# +1440 % 1440 handles midnight wraparound for both offsets.
mount_dicts=""
unmount_dicts=""

for s in "${mins[@]}"; do
  mount_min=$(((s - 2 + 1440) % 1440))
  unmount_min=$(((s + 1) % 1440))
  mount_dicts+=$'\n'"$(hm_dict "$mount_min")"
  unmount_dicts+=$'\n'"$(hm_dict "$unmount_min")"
done

# Write the mount LaunchDaemon plist. StartCalendarInterval entries are
# generated from the times derived above. RunAtLoad is false so the daemon
# only fires at the scheduled times, not on boot.
cat > "$MOUNT_PLIST" <<EOFPL
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key><string>com.tmdock.mount</string>

    <key>ProgramArguments</key>
    <array>
      <string>/bin/zsh</string>
      <string>$MOUNT_SCRIPT</string>
    </array>

    <key>StartCalendarInterval</key>
    <array>$mount_dicts
    </array>

    <key>RunAtLoad</key><false/>
    <key>StandardOutPath</key><string>/tmp/com.tmdock.mount.out</string>
    <key>StandardErrorPath</key><string>/tmp/com.tmdock.mount.err</string>
  </dict>
</plist>
EOFPL

# Write the unmount LaunchDaemon plist, structured identically to the mount
# plist but using the unmount script and its derived fire times.
cat > "$UNMOUNT_PLIST" <<EOFPL
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key><string>com.tmdock.unmount</string>

    <key>ProgramArguments</key>
    <array>
      <string>/bin/zsh</string>
      <string>$UNMOUNT_SCRIPT</string>
    </array>

    <key>StartCalendarInterval</key>
    <array>$unmount_dicts
    </array>

    <key>RunAtLoad</key><false/>
    <key>StandardOutPath</key><string>/tmp/com.tmdock.unmount.out</string>
    <key>StandardErrorPath</key><string>/tmp/com.tmdock.unmount.err</string>
  </dict>
</plist>
EOFPL

# LaunchDaemon plists must be owned by root:wheel with mode 644,
# otherwise launchd will refuse to load them.
chown root:wheel "$MOUNT_PLIST" "$UNMOUNT_PLIST"
chmod 644 "$MOUNT_PLIST" "$UNMOUNT_PLIST"

echo "Wrote:"
echo "  $MOUNT_PLIST"
echo "  $UNMOUNT_PLIST"

Then make executable:

sudo chmod 755 /usr/local/sbin/tm-dock-generate-plists.sh

Step 8 — Generate LaunchDaemons

Run the script generated in Step 7:

sudo /usr/local/sbin/tm-dock-generate-plists.sh

Step 9 — Load LaunchDaemons

Unregister old LaunchDaemons (important if updating them) and load new ones:

sudo launchctl bootout system /Library/LaunchDaemons/com.tmdock.mount.plist 2>/dev/null || true
sudo launchctl bootout system /Library/LaunchDaemons/com.tmdock.unmount.plist 2>/dev/null || true

sudo launchctl bootstrap system /Library/LaunchDaemons/com.tmdock.mount.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.tmdock.unmount.plist

The bootout commands are harmless on their first invocation (before you’ve registered any daemon to launchd) and are critical to run before bootstrap for any future invocations when you want to reload the plist files, which you would need to do if updating the backup schedule.

Step 10 — Configure TimeMachineEditor

Open TimeMachineEditor and set the same times as in times.conf, with the “Calendar Intervals” setting:

After all these steps, now the disk will:

Mount 2 minutes before the scheduled backup
Run its Time Machine backup
Unmount immediately after completion

Step 11 - disable “standby” while on AC power (optional?)

So, I have marked this as “optional?” because it was necessary for me to eliminate all remaining mystery random spin-ups (even with the HDD remaining unmounted…), but I never really identified if this was something unique to my specific dock and/or HDD or if this was a more general thing.

Basically, even after all of the above, I was still having cases where I’d hear the HDD spin up randomly in the middle of the night when the MacBook was set to Sleep, sometimes even flashing on my external displays, despite ensuring the HDD was ejected/unmounted before setting the computer to Sleep. This weirdness would also sometimes leave me with “Disk not properly ejected” notifications when I next got back onto my MacBook, or even with the HDD mysteriously mounted.

The culprit was identified as the “standby” power mode. So, this is more of a separate power management behavior issue rather than something related to Time Machine or the LaunchDaemons here at all.

The solution was simply to disable standby (standby 0) only while on AC power (-c):

sudo pmset -c standby 0

After setting this, I no longer had any random unexpected HDD spin-ups while the MacBook should otherwise be set to Sleep.

If you’re curious about your own standby power setting, you can execute:

pmset -g | grep standby

If you ever need to restore it default behavior later:

sudo pmset -c standby 1

Step 12 — Manual mount Shortcut (optional but strongly recommended)

This step creates the Shortcut that you can simply run in macOS to manually mount the HDD without needing to unplug and replug anything physically, very useful if you ever want to mount it to just inspect the Time Machine backups or restore any files.

Create:

sudo nano /usr/local/sbin/tm-dock-manual-mount.sh

Paste:

#!/bin/zsh
set -euo pipefail

TM_APFS_UUID="PUT_YOUR_APFS_VOLUME_UUID_HERE"
PASSFILE="/usr/local/etc/tm-dock/passphrase"
VOL_PATH="/Volumes/TimeMachine"

[[ -d "$VOL_PATH" ]] && exit 0

# Unlock (if locked). If already unlocked, this prints an error; we ignore it.
# Note: unlocking may mount automatically, but not always.
/usr/sbin/diskutil apfs unlockVolume "$TM_APFS_UUID" -stdinpassphrase < "$PASSFILE" >/dev/null 2>&1 || true

# If still not mounted, determine the current disk identifier (e.g. disk7s2) and mount it.
if [[ ! -d "$VOL_PATH" ]]; then
  DISK_ID="$(/usr/sbin/diskutil apfs list | /usr/bin/awk -v uuid="$TM_APFS_UUID" '
    $0 ~ uuid { getline; if ($1 == "APFS" && $2 == "Volume" && $3 == "Disk" && $4 ~ /^\(Role\):$/) { print $5; exit }
             else if ($1 == "APFS" && $2 == "Volume" && $3 == "Disk") { print $4; exit } }
  ')"

  # Fallback parsing (more robust): look for "APFS Volume Disk (Role): diskXsY"
  if [[ -z "${DISK_ID:-}" ]]; then
    DISK_ID="$(/usr/sbin/diskutil apfs list | /usr/bin/awk -v uuid="$TM_APFS_UUID" '
      $0 ~ uuid { found=1 }
      found && /APFS Volume Disk/ { for (i=1;i<=NF;i++) if ($i ~ /^disk[0-9]+s[0-9]+$/) { print $i; exit } }
    ')"
  fi

  if [[ -n "${DISK_ID:-}" ]]; then
    /usr/sbin/diskutil mount "$DISK_ID" >/dev/null 2>&1 || true
  fi
fi

Make sure to replace "PUT_YOUR_APFS_VOLUME_UUID_HERE" with your UUID from Step 1.

Make executable:

sudo chmod 755 /usr/local/sbin/tm-dock-manual-mount.sh

Create a Shortcut that runs:

sudo /usr/local/sbin/tm-dock-manual-mount.sh

If you’ve never made a shortcut before (and oh boy you’re in for a wold of automation goodness on macOS/iOS), then you can find the Shortcuts app via Spotlight, click the “+” to make a new Shortcut, search for the “Run Shell Script” action, and configure it as shown below, making sure to enable “Run as Administrator”.

Then, you can access the Shortcut however you please. I personally like having it in the Shortcuts icon in my menu bar, but you can always run it directly from the Shortcuts app, via Siri, via a dock icon, or even a custom keyboard shortcut.

Updating the scheduled times

If you ever want to change your backup schedule, then you essentially only need to repeat Steps 3, 8, 9, and 10. That is, you just need to:

update the times listed in /usr/local/etc/tm-dock/times.conf (edit with sudo nano)
and in TimeMachineEditor to whatever new times you want, then
run the /usr/local/sbin/tm-dock-generate-plists.sh script (run with sudo) to generate the updated LaunchDaemon files,
and finally reload them with:

sudo launchctl bootout system /Library/LaunchDaemons/com.tmdock.mount.plist 2>/dev/null || true
sudo launchctl bootout system /Library/LaunchDaemons/com.tmdock.unmount.plist 2>/dev/null || true

sudo launchctl bootstrap system /Library/LaunchDaemons/com.tmdock.mount.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.tmdock.unmount.plist

Serenity achieved

What began as a naive “surely it can’t be that hard to keep my HDD quiet except when minimally necessary?” question spiraled into a full-on macOS troubleshooting saga, but I am so pleased to have finally fully solved this issue, to have my noisy HDD spin-ups only occurring on a fully deterministic basis. While I admittedly think this really should be easier to solve for the average user rather than solely for power-users comfortable with not only terminal commands but with invoking sudo permissions too, I am glad it is at least solvable without needing to resort to any horrifically janky solutions. (At some point I was contemplating the viability of having some physical switch on my desk to toggle the HDD’s connection to the dock or to power…)

I will add that, as of writing this, I am no expert in macOS at all; I’m still very fresh to macOS but with a fair amount of professional experience operating on remote Linux systems. This was a huge learning experience for me, and I don’t necessarily regret being forced to learn all sorts of nitty-gritty details of pmset power management and states, launchctl LaunchDaemons, diskutil disk utility mounting with APFS-encrypted volumes, tmutil Time Machine utility, and other bits and pieces, as “learning through practical projects” is my preferred style, and maybe I’ll be able to apply this knowledge to future automation and scheduling tasks. That said, I really hope any future problems I want to solve on macOS will have much simpler solutions. 😅

A primer on construction, selection, and filtering of neutron and gamma-ray event cones, applied to simple back projection imaging

2026-03-11T00:00:00-07:00

[Direct link to full PDF of this article]

Abstract: This document details the methodology for building, selecting, and filtering neutron and gamma-ray event cones for imaging in the context of the detector system composed of long-form-factor organic plastic scintillators developed in the NOVO project and as utilized in the ng-imager Python toolkit for neutron and gamma-ray imaging. A summary of the simple back projection method implemented is provided too. This work employs PHITS simulations of a simple six-bar detector array, providing data free of resolution effects and accompanied by extra “Monte Carlo truth” information to benchmark reconstruction method results against.

An experimental data pipeline, such as in the NOVO project, can be generally divided into five distinct main stages, requiring analysis procedures to progress from one stage to the next:

Raw detector data (integrated charges, timestamps, waveforms)
Physical quantities derived (energy deposited, interaction position, times)
Coincident events (physical quantities from double neutron / triple gamma-ray interaction coincidences, fulfilling various criteria)
Event cones (vertex, direction, half-angle)
Imaging (projection of event cones onto an imaging plane)

This work outlines the procedure for constructing neutron and gamma-ray event cones provided derived physical quantities from coincident events. Each particle contains some unique challenges in this regard. For neutrons, this includes determining the reaction’s nature (elastic scattering, inelastic scattering, or other nonelastic) and target (proton or carbon nucleus). For gamma rays, this includes determining the correct sequencing of the three reactions in coincidence to yield the most probable valid event cone. A condensed outline of the analytically-solved simple back projection imaging implementation used in ng-imager in “scan” mode is also provided.

Simulation setup

The simulations were conducted in PHITS (v3.29) [1] and set up as follows. Six parallel plastic scintillator bars of dimensions 12$\times$12$\times$140 mm$^3$ were arranged in a $3\times2$ pattern with a center-to-center spacing of 60 mm (48 mm gap width) to account for some real-world clearance for mounting hardware. The simulated design is pictured in Figure 1.

Figure 1. Simulated plastic scintillator array

In all neutron simulations, the source was an isotropic, 14.1 MeV monoenergetic neutron source placed 100 cm from the front face of the detector array and in line with its center. From the perspective of Fig. 1, the neutrons enter the picture from the lower left to then strike the array. While 100 million histories were simulated, the source was modeled as a cone with angle $\theta=5.5^\circ$ pointed toward the array, allowing simulation of only neutrons bound for the array. Owing to the cone’s 0.0289 sr solid angle, this was the equivalent of simulating 43.4 billion ($4.344\times10^{10}$) deuterium-tritium (DT) fusion events.

Provided the array is placed far enough from the source such that this source-to-array cone is only a few degrees wide, this monoenergetic assumption is valid. Likewise, a DT source is close enough to being completely isotropic for this assumption to hold as well.

Gamma-ray simulations employed a 2 MeV monoenergenic gamma-ray source otherwise identical to the neutron source described above. 1 billion histories (or an effective 434 billion emissions, corrected for solid angle) were simulated.

In a list mode fashion, neutron and gamma-ray interactions were recorded to “dump” files, and in post multiple scattering events were constructed, sequenced, and put into a new list for further tallying and analysis. Furthermore, neutron simulations included two tallies for interactions in the bars: one of only elastic scattering reactions and one for all nuclear reactions (including elastic scattering).

For the sake of events considered valid for imaging, only neutron double coincidences where each scatter deposited at least 500 keV were considered. This threshold was set to 100 keV for each of the three gamma-ray interactions.

Nuclear data libraries (JENDL-4.0) were employed for transport of neutrons $<20$ MeV instead of models (nucdata=1), and the event generator mode was enabled (e-mode=2) for accurate event-wise reaction data.

Cone construction and selection

Ultimately, each cone is defined by a vertex coordinate $O=(x_O,y_O,z_O)$ with an axis/direction vector $\hat{D}=\langle u,v,w \rangle$ and interior half-angle $\theta$. These are derived from the following physical quantities extracted from experimental data (and emulated in simulation):

For neutron events:
- 1^st scatter location $X_1=(x_1,y_1,z_1)$, time $t_1$, and energy deposited $E_{\mathrm{dep},1}$
- 2^nd scatter location $X_2=(x_2,y_2,z_2)$ and time $t_2$
For gamma-ray events:
- 1^st scatter location $X_1=(x_1,y_1,z_1)$ and energy deposited $E_{\mathrm{dep},1}$
- 2^nd scatter location $X_2=(x_2,y_2,z_2)$ and energy deposited $E_{\mathrm{dep},2}$
- 3^rd scatter location $X_3=(x_3,y_3,z_3)$ (and energy deposited $E_{\mathrm{dep},3}$)

The process of creating event cones from this experimental/simulated data is referred to here as “cone construction.” For both neutrons and gamma rays, the vertex coordinate $O$ is the location of the first scatter, the direction vector $\hat{D}$ is the unit vector pointing from the second scatter to the first scatter, and the cone half-angle $\theta$ is the scattering angle of the first collision. The $\theta$ calculation differs for neutrons and gamma rays.

Neutron cone half-angle $\mathbf{\theta}$

For neutrons scattering elastically, the scattering angle in the center-of-mass (CoM) reference frame $\theta_{\mathrm{n,CoM}}$ is defined, with recoil nucleus energy $E_{\mathrm{recoil}}$, incident neutron energy $E_\mathrm{n}$, and ratio of the recoil nucleus’s mass to the neutron mass $A$ (Equation (1)), as [2]:

\begin{equation} A = \frac{m_\mathrm{recoil}}{m_\mathrm{n}} \label{atomicmass} \end{equation}

\begin{equation} \theta_{\mathrm{n,CoM}} = \arccos\left( 1 - \left( \frac{E_\mathrm{recoil}}{E_\mathrm{n}} \cdot \frac{(1+A)^2}{2A} \right) \right) \label{theta_CoM} \end{equation}

Experimentally, $E_\mathrm{recoil}$ is taken to be the energy deposited in the first scattering reaction $E_{\mathrm{dep},1}$. The scattered neutron’s energy $E_\mathrm{n}^\prime$ is determined with its time-of-flight (Equation (4)) and flight distance $s_\mathrm{n}$ (Cartesian distance between $X_1$ and $X_2$, Equation (5)) relativistically, with speed of light $c$, as:

\begin{equation} E_\mathrm{recoil} = E_{\mathrm{dep},1} \label{neutron_Erecoil} \end{equation}

\begin{equation} t_\mathrm{n,ToF} = t_2 - t_1 \label{neutron_tof} \end{equation}

\begin{equation} s_\mathrm{n} = \left| \vec{X_1X_2} \right| = \sqrt{ (x_1-x_2)^2 + (y_1-y_2)^2 + (z_1-z_2)^2 } \label{neutron_flightpath} \end{equation}

\begin{equation} v_\mathrm{n} = \frac{s_\mathrm{n}}{t_\mathrm{n,ToF}} \label{neutron_velocity} \end{equation}

\begin{equation} E_\mathrm{n}^\prime = \left( \sqrt{\frac{1}{1-(v_\mathrm{n}/c)^2}} - 1 \right) m_\mathrm{n}c^2 \label{neutron_KE} \end{equation}

For completeness’s sake, this would be calculated classically (non-relativistically) as shown in Equation (8). As shown in Figure 2, in the neutron energies relevant to proton therapy, the classical approach to calculating $E_\mathrm{n}$ deviates quite notably from the relativistic approach.

\begin{equation} E_\mathrm{n,\,non\text{-}relativistic}^\prime = \frac{1}{2}m_\mathrm{n}v_\mathrm{n}^2 \label{neutron_KE_classic} \end{equation}

Figure 2. What classical mechanics would predict a neutron’s kinetic energy to be at each true kinetic energy (as calculated relativistically using the same neutron velocity).

The incident neutron energy $E_\mathrm{n}$ is simply the sum of the scattered neutron energy $E_\mathrm{n}^\prime$ and the energy of the recoil product $E_\mathrm{recoil}$ (Equation (9)). Note that this assumption fails when the recoil product does not deposit all of its energy in the detection volume (and also fails when the reaction is nonelastic).

\begin{equation} E_\mathrm{n} = E_\mathrm{n}^\prime + E_\mathrm{recoil} \label{neutron_incident_KE} \end{equation}

The neutron scattering angle in the lab frame $\theta_{\mathrm{n,lab}}$ is found from $\theta_{\mathrm{n,CoM}}$ with:

\begin{equation} \theta_{\mathrm{n,lab}} = \arctan\left( \frac{\sin(\theta_{\mathrm{n,CoM}})} {\cos(\theta_{\mathrm{n,CoM}}) + (1/A)} \right) \label{theta_lab} \end{equation}

Strictly speaking, the $\mathop{\mathrm{atan2}}$ function, rather than $\arctan$, is used here as the desired output angle should range from 0 to $\pi$, not $-\pi/2$ to $\pi/2$. To avoid this, $\theta_{\mathrm{n,lab}}$ may also be equivalently expressed as [3]:

\begin{equation} \theta_{\mathrm{n,lab}} = \arccos\left( \frac{1 + A\cos(\theta_{\mathrm{n,CoM}})} {\sqrt{A^2 + 2A\cos(\theta_{\mathrm{n,CoM}}) + 1}} \right) \label{theta_lab_v2} \end{equation}

For the ease of discussion, when mentioning the neutron cone half-angle $\theta$ with no subscript, it is in reference to the lab frame $\theta_{\mathrm{n,lab}}$.

Gamma-ray cone half-angle $\mathbf{\theta}$

For a gamma ray undergoing solely Compton scattering in its first two interactions, the scattering angle of the first collision $\theta_{\gamma,1}$ can be found via Compton scattering kinematics [4]. First, the scattering angle of the second interaction $\theta_{\gamma,2}$ is calculated from the vectors $\overrightarrow{X_1X_2}$ and $\overrightarrow{X_2X_3}$ connecting the first to second scatter location and the second to third scatter location, respectively.

\begin{equation} \theta_{\gamma,2} = \arccos\left( \frac{ \vec{X_1X_2}\cdot\vec{X_2X_3} }{ |\vec{X_1X_2}||\vec{X_2X_3}| } \right) \label{eq:theta_gamma2} \end{equation}

Assuming that the energy lost by the gamma ray in each scatter is equal to the energy deposited $\Delta E_i = E_{\mathrm{dep},i}$ (true of Compton scattering events whose recoil electrons do not escape), the initial incident gamma ray’s energy $E_\gamma$ can be calculated as:

\begin{equation} E_\gamma = \Delta E_1 + \frac{1}{2} \left( \Delta E_2 + \sqrt{ \Delta E_2^2 + \frac{4\Delta E_2 m_e c^2}{1-\cos(\theta_{\gamma,2})} } \right) \label{eq:Egamma} \end{equation}

Then, the gamma ray’s energy after the first scatter $E_\gamma^\prime$ is simply the difference between this initial energy and the amount lost in the first scatter:

\begin{equation} E_\gamma^\prime = E_\gamma - \Delta E_1 \label{eq:E_gamma_prime} \end{equation}

And the first scattering angle $\theta_{\gamma,1}$ is:

\begin{equation} \theta_{\gamma,1} = \arccos\left( 1 + m_ec^2 \left[ \frac{1}{E_\gamma} - \frac{1}{E_\gamma^\prime} \right] \right) \label{eq:theta_gamma} \end{equation}

And thus the gamma-ray cone half-angle $\theta$ is found. It should be noted that all gamma-ray math is already using relativistic mechanics.

There are complications for both neutron and gamma-ray cone half-angle calculations. For neutrons, determining whether a scatter was with a proton or carbon nucleus is a challenge, and assuming that all interactions are elastic scatters is not always valid. In nonelastic reactions, the change in the neutron’s energy is no longer always equal to the energy deposited in the detector. For gamma rays, determining the actual order of the scattering events experimentally is a much greater challenge as the time values are so close to each other as to be unusable for correctly sequencing the scatters. These challenges are addressed in the following subsections.

Neutron cone generation

The vast majority of neutron scattering events fall into one of the following categories:

Elastic scatter with protons
Elastic scatter with carbon nucleus
Inelastic scatter / nuclear reaction with carbon nucleus

Other channels exist (such as ¹H(n,$\gamma$)) but have significantly lower cross section, and that last category of nonelastic reactions with carbon can be further broken down into individual reaction channels. For simplicity’s sake, we’ll begin with the simulated results where only elastic scattering events were tallied. Figure 3 shows a comparison of the calculated values of $\theta$ versus the Monte Carlo (MC) true values of $\theta$, assuming all reactions are with either protons or carbon nuclei.

(a) assuming all recoils with protons

(b) assuming all recoils with carbon

Figure 3. Comparison of calculated and MC truth scattering angles $\theta$ when assuming all scatters occur with the same target nucleus (only including elastic scattering events).

There are two clear lines in each, with one matching the MC truth values, letting us know these lines do indeed correspond to elastic scattering with protons and carbon nuclei. (One should note that Figure 3b only looks “cleaner” owing to the fact that all protons of $\theta_{\mathrm{MC truth}}\gtrapprox0.56$ end up with an $\arccos$ argument in Equation (2) $> \lvert 1 \rvert$, resulting in a NaN value of $\theta$ which propagates through the remainder of the calculation and is ultimately not plotted.) This presents a challenge though, without knowledge of the MC truth, how are we to decide whether a given event corresponds to a proton or carbon scattering?

In most real situations, one should have some approximate idea of where the source is located. In this simulated case, it is a true point source, but at far enough distances or small enough sources, this assumption is transferable to the real world. Thus, we can simply calculate $\theta$ for both the proton and carbon scatters, compare those angles to the angle $\theta_\mathrm{est}$ between $\hat{D}$ and the vector from the cone vertex $O$ to the estimated source location, and pick whichever is closest. This procedure yields Figure 4.

Figure 4. Comparison of calculated and MC truth scattering angles θ when calculating angles for both proton and carbon scatters and picking the one closest to the estimated angle between the direction vector and the vector from the vertex to source location.

At least in this case, the method works quite well! This discrimination approach certainly would need to be adjusted in more clinical scenarios where the source term is distributed and the detector array would be positioned closer to it. Perhaps such discrimination could incorporate additional experimental data.

Moving on, Figure 5 shows this methodology applied to the simulation results containing all nuclear reactions, not just elastic scattering.

Figure 5. Same as Figure 4 but for all nuclear interactions, not just elastic scattering.

While the elastic scattering line is still very prominent, some new interesting features emerge from inelastic and other nonelastic reactions¹ with carbon. One approach would be to simply filter out these new events by rejecting $\theta$ values which exceed some absolute or fractional difference from the estimated $\theta_\mathrm{est}$. This is not the most future-proof approach for the aforementioned reasons, but it is simple and quite effective, as illustrated for a 5% fractional difference threshold in Figure 6.

Figure 6. Same as Figure 5 but eliminating all events whose calculated θ closest to θ_est deviates by more than 5% from it. Of these 10322 event cones, 7501 were attributed to proton recoils and 2821 to carbon recoils.

This approach / last step is more for illustration and should NOT used experimentally as it also “forces the right answer”, i.e., we only image the cones that yield the image we’re expecting. In experimental work, the fairer angular constraint to place on the cones is that they be pointing in the general $2\pi$ direction toward the source. Thus, the net effect of including all of these nonelastic scattering events is added noise/blurring in the neutron images.

It may be possible that nonelastic events can be considered in some more clever way, rather than just being a source of noise, but this seems unlikely to be feasible experimentally, as the experimental data is not remotely as clean as the MC truth data presented here, making it seemingly impossible to distinguish elastic and nonelastic interactions in any practical and fair way. Still, with good pulse shape discrimination, it may be possible to reliably distinguish proton from carbon recoils as well as the alphas from carbon breakup reactions.

Gamma-ray cone generation

The primary challenge in generating event cones for gamma rays is event sequencing. Owing to the tight bar spacing in NOVO, time resolution of the measurement apparatus, and light-speed velocity of gamma rays, the experimentally produced timestamps for gamma rays are effectively meaningless for determining the actual order of the three coincident interactions. Thus, some other approach must be developed for determining the true, or at least most probable, sequencing of the three interactions.

Fortunately, there are only six possible arrangements for three interactions (123, 132, 213, 231, 312, and 321), and all combinations can be tested. When solving Equation (15) for $\theta_1$, the kinematically nonsensical arrangements will make themselves evident by yielding a value outside of $[-1,1]$ for the argument of $\arccos$ or returning a $\theta_1$ of 0. Eliminating these leaves only the potentially valid sequences of the three interactions. Plotting all these calculated $\theta$ values against the Monte Carlo true $\theta$ for the simulation of a 2 MeV gamma-ray source described earlier yields Figure 7.

Figure 7. Comparison of calculated and MC truth gamma-ray scattering angles θ for all valid gamma-ray event cones in each event

Here, a clear line is present consisting of the correctly calculated $\theta$ values, but they are in a sea of faulty cones. While one could image all of these cones, it would be more computationally expensive and produce a noisier image. The next logical step would be, rather than selecting all potentially valid cones for each event, to select the “best” cone among the valid ones for each event. At present, in a similar fashion to the neutron cone generation, all valid cone angles are compared to the angle $\theta_\mathrm{est}$ between $\hat{D}$ and the vector from the cone vertex $O$ to the estimated source location, and whichever is closest is picked as the “best” cone. This is shown in Figure 8.

Figure 8. Same as Figure 7 but only selecting for each event the “best” cone whose θ is closest to the estimated angle θ_est between the direction vector and the vector from the vertex to source location.

This eliminates almost all errant $\theta$ values. Given that the remaining calculated $\theta$ deviating from the Monte Carlo true $\theta$ are already the “best” of the possible valid $\theta$, it makes sense to conclude that these events correspond to situations where the earlier assumptions (that all interactions are Compton scattering and that all recoil electrons stop in the bars) prove to be false. This, however, is only a very tiny minority of events. Similar to the neutron cone generation method, an arbitrary maximum deviation from an estimated $\theta_\mathrm{est}$ can be enforced and is illustrated for a 5% fractional difference threshold in Figure 9. (Again, this final cleaning step should NOT done for experimental data.)

Figure 9. Same as Figure 8 but eliminating all events whose calculated θ closest to θ_est deviates by more than 5% from it.

With this approach, accurate event cones can be constructed for nearly all gamma-ray events. However, other approaches for selecting the “best” cones may need to be considered in the future depending on the magnitude of measurement uncertainties.

Imaging

What I am calling “imaging” is the process of projecting a constructed event cone—with a vertex coordinate $O=(x_O,y_O,z_O)$, axis/direction unit (column) vector $\hat{D}=\langle u,v,w \rangle^\intercal$, and interior half-angle $\theta$—onto a two-dimensional plane to create an image. Mathematically, this cone is described as the set of all points $X = [x_X,y_X,z_X]^\intercal$ which satisfy [5]:

\begin{equation} (X-O)^\intercal M (X-O) = 0 \label{eq:cone} \end{equation}

Where:

\begin{equation} M = \hat{D}\hat{D}^\intercal - \cos^2(\theta)I_{3\times3} \label{eq:cone_M} \end{equation}

For simplicity, let:

\begin{gather} U = X - O = \begin{bmatrix} x_X - x_O \\ y_X - y_O \\ z_X - z_O \end{bmatrix} = \begin{bmatrix} x_U \\ y_U \\ z_U \end{bmatrix} \label{eq:simplify} \end{gather}

And substitute \ref{eq:simplify} into \ref{eq:cone} and expand:

\begin{gather} \begin{bmatrix} x_U & y_U & z_U \end{bmatrix} \begin{bmatrix} M_{00} & M_{01} & M_{02} \\ M_{10} & M_{11} & M_{12} \\ M_{20} & M_{21} & M_{22} \end{bmatrix} \begin{bmatrix} x_U \\ y_U \\ z_U \end{bmatrix} = 0 \label{eq:cone_matrix} \end{gather}

Multiplying the matrices gives us a second-order polynomial describing the cone:

\begin{align} x_U^2M_{00} + y_U^2M_{11} + z_U^2M_{22} + x_Uy_U(M_{01}+M_{10}) \notag\\ {} + y_Uz_U(M_{12}+M_{21}) + x_Uz_U(M_{02}+M_{20}) &= 0 \label{eq:cone_poly} \end{align}

The mathematics is simpler if forcing the imaging plane to be perpendicular to a coordinate axis. While the ng-imager implementation allows arbitrary plane orientations, the same underlying mathematics applies after expressing coordinates in the plane’s basis, which is assumed for the sake of simplifying this discussion. The image generation process is as follows:

Define an imaging plane/window with two points, $P_{BL}$ and $P_{TR}$, denoting the bottom-left and top-right corners of the imaging window, which have one matching coordinate variable whose axis the plane lies perpendicular to. This will set the corresponding element of $X$ to a constant.
Define the horizontal and vertical spatial pixel resolutions (e.g., 1 mm). Extract from $P_{BL}$ and $P_{TR}$ the horizontal and vertical bounds of the imaging window. Create 1D arrays for the bin centers and bin edges for both the columns and rows of the image, and create a 2D array $G$ of zeros corresponding to the image rows and columns (referred to as the image array).
For each cone:
1. Initialize two empty lists for storing horizontal and vertical coordinates, $L_H$ and $L_V$, respectively.
2. Loop through the horizontal bins and in each:
  1. Set the corresponding element of $X$ to the bin center.
  2. Calculate the two known elements of $U$ provided $X$.
  3. Solve Equation (20) for the remaining unknown element of $U$.
  4. Substitute the real $\mathbb{R}$ root(s) found into Equation (18) to obtain the value(s) of last unknown element of $X$.
  5. Append the found value(s) for the $X$ elements corresponding to the image’s horizontal and vertical axes to their respective lists $L_H$ and $L_V$.
3. Repeat the same loop as above but for the vertical bins.
4. Score/tally the coordinate pairs in $L_H$ and $L_V$ in a 2D histogram of the same dimensions as the 2D image array $G$ defined earlier and using the bin edges defined earlier. Set all nonzero entries in the histogram to 1.
5. Add the histogram to the 2D image array $G$.
Plot the 2D image array $G$ to show the obtained image.

There are a few notes to be made about this approach. First, the image window is “scanned” in both the horizontal and vertical directions to ensure that each ellipse (technically, conic section, but usually an ellipse) in the imaging window is “continuous” from pixel to pixel. Additionally, the current method is only counting pixels as hit or not hit. Rather than this binary approach, a fancier approach could incorporate the length of the ellipse segment within each pixel and weight them accordingly but is not explored here.

Using this method, Figure 10a is generated for a 1 mm$^2$ pixel resolution and 100 mm wide and tall image window centered on the source location (0,0,0), using the neutron event cones generated earlier, employing results from all nuclear reactions but filtering out rejected events (see Figures 5 vs. 6 earlier). The same image but for the gamma-ray event cones generated earlier (see Figure 9) is shown in Figure 10b.

(a) 14.1 MeV neutron image

(b) 2 MeV gamma-ray image

Figure 10. The images generated from neutron and gamma-ray events simulated earlier.

As a bit of a fun simulation and to test a non-trivial source, next was simulated and imaged a ring-shaped source centered at (0,30,40) with inner radius 48 mm and outer radius 52 mm, isotropically emitting 14.1 MeV neutrons. Its image is shown in Figure 11 and verifies the imaging approach is working as intended.

Figure 11. Image of a neutron ring-shaped source.

Effect of using “realistic” interaction coordinates

In the images thus far, all interaction coordinates have used the exact MC true values. In the real experiment, we must assume that every interaction happens at the center of the bar in two dimensions and only has granularity along the depth of interaction (DOI) axis. Figure 12 shows the exact same image as earlier with Figure 10 but only using the exact MC coordinate for the DOI axis and using the location of the bar’s centerline for the other two dimensions, keeping everything else unchanged.

(a) 14.1 MeV neutron image

(b) 2 MeV gamma-ray image

Figure 12. Same as Figure 10 but using the bar centerline coordinates for each interaction position for two of the coordinate axis values and the MC true value along the DOI axis.

To make matters worse, experimental position resolution / uncertainties in the DOI axis actually tend to be worse / greater than those in the bar’s two cross sectional directions. Fortunately though, with enough statistics these issues can, at least to some degree, be overcome.

Conclusions

A consistent methodology for constructing, selecting, and filtering event cones for both neutrons and gamma rays has been outlined here. Provided “pure” Monte Carlo data, the methods perform very well. The analytic, raster scanning-esque implementation of simple back projection employed here performed very well, producing consistently high-quality images.

References

[1] T. Sato, Y. Iwamoto, S. Hashimoto, T. Ogawa, T. Furuta, S. Abe, T. Kai, P.-E. Tsai, N. Matsuda, H. Iwase, N. Shigyo, L. Sihver, and K. Niita, “Features of Particle and Heavy Ion Transport code System (PHITS) version 3.02,” Journal of Nuclear Science and Technology, vol. 55, no. 6, pp. 684-690, 2018. doi: 10.1080/00223131.2017.1419890

[2] R. Fitzpatrick, Newtonian Dynamics. The University of Texas at Austin (online), 2008.

[3] J. A. Bernard, 22.05 Neutron Science and Reactor Physics, Fall 2006 (online), 2006.

[4] I. Meric, E. Alagoz, L. B. Hysing, T. Kögler, D. Lathouwers, W. R. B. Lionheart, J. Mattingly, J. Obhodas, G. Pausch, H. E. S. Pettersen, H. N. Ratliff, M. Rovituso, S. M. Schellhammer, L. M. Setterdahl, K. Skjerdal, E. Sterpin, D. Sudac, J. A. Turko, and K. S. Ytre-Hauge, “A hybrid multi-particle approach to range assessment-based treatment verification in particle therapy,” Scientific Reports, vol. 13, no. 1, p. 6709, 2023. doi: 10.1038/s41598-023-33777-w

[5] D. Armstrong, “Where is the cone?” arXiv:1708.07093, 2018. arXiv:1708.07093

For clarification, I mean inelastic scattering as reactions just leaving the target nucleus in an excited state and nonelastic reactions as inelastic scattering plus all other reactions resulting in everything else aside from just a neutron and the original target nucleus. ↩

Choosing macOS after a lifetime on Windows

2026-03-01T00:00:00-08:00

A younger version of myself would have scoffed and even laughed at the thought of ever buying a Mac. Little did he know that over the years all of his arguments against Apple for his computing needs would either be flipped upside down or fade into irrelevancy—and that Microsoft would be the one to provide him that final push to buy a MacBook.

I grew up in a Windows household, introduced to computers from a young age. I played the original Putt-Putt game on a Compaq PC running Windows 95, spent countless hours in Microsoft Paint recoloring Pokemon sprites on our Windows XP machine, fell in love with PC gaming on a hand-me-down laptop running Windows Vista (later upgrading to a more capable Windows 7 laptop), and eventually built my own desktop PC with Windows 10. I became more and more a power-user over the years, especially once my grasp on coding and scripting was developed in my university years. I’d burned into muscle memory a bunch of keyboard shortcuts and even a moderate list of Alt codes—after all, I do love my em and en dashes (Alt + 0151/0150). I became comfortable in the CME.exe terminal, edited Windows registry keys to fix bugs/annoyances, used robocopy for offline file syncing, and generally became acquainted with lots of power-user features, utilities, and programs through my professional work. I felt good about my mastery over Windows, after all, the professional world runs on Windows, right?

Simultaneously, I had been nursing a relationship with the Apple ecosystem most of this time. For Christmas in eigth grade I received an iPod classic, and this set in motion how I would manage my music to this very day, curating my own collection of MP3 files and playlists, a holdout in today’s era of music streaming. When I went off to university, I got my first smartphone, an iPhone 4, the natural choice given my music lived in iTunes. To some degree, I had initialy felt a bit like a hostage, that I had to use an iPhone owing to insisting on having my music in iTunes and not wanting to mess with other music management solutions. That feeling, fortunately, mostly faded before too long owing to actually liking the mobile experience on iOS, though I did harbor some envy for the extra customization, control, and freedoms of Android.

Even then though, I would never consider buying a Mac. It was simply unfathomable. Why would I pay a lot more money for a machine with worse performance, second-class to absent support for many programs and games especially, and on a relatively niche operating system with hardly any footprint in the professional world of engineering? Furthermore, I saw value in being able to customize everything, in both hardware and software. Maybe this made me a hypocrite for using iOS over Android, but at least on mobile there were plenty more arguments, in my view, for choosing Apple (device build quality, OS polish and consistency, software support longevity, etc.). In the personal computing realm, the only choices were Microsoft Windows, Apple macOS, and Linux, and Windows was obviously the right choice. And this is how I continued thinking throughout my twenties.

In the summer before my final year of undergrad, I had an internship where I was provided an iMac to use, and I remember just being so frustrated with it. I resented Apple for so many Mac-isms and for, what seemed to me, unintuitive differences in design whose only purpose was to be different. In all fairness, I still hold many of those opinions (looking at you, reversed “natural” scrolling), and being cursed with a Magic Mouse—a design and erginomic nightmare so bad as to be comical—probably contibuted a fair amount toward my tainted view of Macs and Apple’s design philosophy. At the time, I failed to critically reflect on why I had been given a Mac in the first place, for a job involving writing, compiling, and executing Fortran and C++ code.

I did get a taste of Linux in my university years too, though mostly through PuTTY SSH windows, primarily for utilizing the department’s high-performance computing resources for extensive radiation transport simulations. I started understanding Linux’s place in the professional world and its utility. In my first summer of grad school I had the joy of trying to compile Fortran code on a Windows machine, realizing just how much more of a pain it was than in my undergraduate Fortran course where I just SSH’d into one of the department’s Linux machines. For the particle experiments in my PhD work, we used CERN’s ROOT software for data analysis, where I was stuck using an old version owing to deprecation of Windows support. These were my first professional tastes of Windows being the second-class citizen and challenging my preconceptions of the professional ubiquity of Windows.

From grad school onward, I was content with my Windows + iOS setup. Neither were perfect, but I felt like I had made the right choice for my personal needs and priorities. My desktop PC served me well, satisfying my gaming needs and providing a smooth and functional experience. Time continued to pass. As a young adult, I became more aware of the news and formed opinions about how the world should be. I aligned with open source ideals and right-to-repair initiatives (a place where Apple holds a notorious reputation), and more broadly I developed values of personal privacy and data security. Amid incremental steps toward a more Orwellian future, Apple has repeatedly demonstrated its resolve on maintaining its principled stance of protecting user privacy even against the strongest adversaries possible (large Western governments), and this has left a strong impression on me.

Even still, I was content with my Windows 10 PC I’d built, and I continued to game on it, even lugging it with me to Japan when I moved there (and later Norway too). However, in those years in Japan, working my first “real job” after my PhD, I started doing a bit more introspection about how I was spending my free time, and specifically on the value of my continuing years-long relationship with World of Warcraft, especially as the game itself had started feeling more like a chore than a fun way to spend time. More broadly, I had begun questioning the value of time spent gaming versus productive or other consumptive hobbies, like reading and watching shows/movies. Regardless, owing to the pandemic keeping me at home for most of my time not at work, gaming remained a part of my life, though in a waning capacity and with a shift back toward console gaming (hello Animal Crossing). It was also in these years that the Cortex podcast, which I’d been a loyal listener of, began chronicling the rise of Apple Silicon computer processors, sparking within me the tiniest bit of intrigue for Mac computers.

By the time life had taken me to Norway and I had fully settled in, my priorities and how I spent my free time were quite different from years past. I no longer found myself gaming at all, aside from occasionally booting up my old Minecraft world for some nostalgia and cathartic building or casually playing various single-player or local-multi-player games on my Nintendo Switch. Instead I was spending my free time reading, watching shows and YouTube edutainment, hiking, and socializing. Thus, my aging gaming PC was largely relegated to web browsing and basic computer tasks.

As time marched on, I started noticing a decline in the quality of my experience on Windows. “Did that pop-up always appear after each update? Wait, where on earth did that Word document just save? What do you mean on OneDrive? You mean I can’t make the default behavior to just save the file locally, that I’m forever plagued with extra clicks to get the more sensible behavior? Why do I need to sign-in to a Microsoft account for this? No, really, I am not interested in the Edge browser. Nor Bing search. Please, I meant no the first three dozen times I told you, and no my mind has not and will not change.” The modern decline of software user experiences to maximally squeeze profits from the user base is a well-documented phenomenon, and Microsoft most certainly has not been immune to it.

And this was all before Copilot and Microsoft’s decision to force-feed its users with its solution-looking-for-a-problem AI agent, intruding everywhere in Windows and Microsoft’s other software products. I was flabbergasted when I first heard of Microsoft’s new Windows 11 ~~spyware~~ “feature” called Recall that constantly records your screen for an AI to study, a blatant and reckless disregard for privacy and security. Even after immense backlash, they have still deployed it—though at least it’s only opt-in by default, for now. The fact that this idea was even humored seriously, let alone developed then deployed, was the writing on the wall for me: my ideals are fundamentally incompatible with where Microsoft wants to take Windows. Furthermore, with Microsoft already claiming up to 30% of their code is being written by AI (and hearing through the grapevine that the target is an even higher percentage to be written by Copilot), I personally lost all faith in the long-term future of Windows, at least for me. This sounds like a recipe for the operating system, which already feels like a patchwork mess (with some functions still only accessible via a decades-old control panel), to just completely implode.

Still, inertia is a very strong force. I may not like Windows 11 on my work laptop, but at least I still have good ol’ Windows 10 on my home PC that I can just keep using, right? Nope! I can’t fault Microsoft for phasing out old operating systems; expecting them to support Windows 10 forever would be unreasonable. But, I can and absolutely will fault them for deprecating tons of perfectly serviceable hardware, including my desktop, by ceasing security updates for Windows 10 and requiring newer hardware for Windows 11, seeking to generate tons of e-waste in an effort just to get people on older hardware to shell out for the shiny new “Copilot+ PCs” on offer, equipped with the latest and greatest user-data-harvesting technology. So, an expiry date of 14 October 2025 had been slapped on my trusty (though aging) home-built PC.

Thus, Microsoft forced my hand. If I wanted to keep using my old desktop PC securely, my only option would be to switch to using Linux on it. While Steam’s recent hardware survey shows a good chunk of gamers have made exactly this choice, I ultimately decided that this was not tenable for me, at least not for my primary machine. While Linux does best align with some of my ideals around software, practically speaking I do not have the time to undertake all of the learning, tinkering, debugging, maintaining, and fixing that would come with a move to Linux, not to mention the fact that there are still widespread software compatibility issues I would have to overcome on Linux. This could still be a fun hobby project for the old desktop one day though. But, the more pressing issue was deciding what kind of new primary computer to buy to replace my nearly-ten-year-old desktop for daily use.

This is the context that ultimately made me question: “Do I even want a Windows 11 machine?” Up until this point, it was a given that I would always use Windows as my primary operating system. But, owing to the ill-will Microsoft had garnered from me in forcing my hand on upgrade timing, together with its consistent display of extreme misalignment with my own values, I was starting to ponder: “Is macOS just my only choice here?” At first, I was really not happy with the pickle I found myself in, trying to just pick the least bad option in front of me. After all, what about all of the reasons I had dismissed Macs previously?

Well, it turns out most of those were moot or wrong now. Computer gaming is the least important it’s ever been to me in my adult life, so there goes that argument for Windows (even with Mac/Linux gaming continuing to make remarkable progress in recent years). While it’s true Apple still charges a premium for its products, I’m also a working adult now and not a poor university student. Furthermore, what that money gets me today is genuinely best-in-class build quality (something Apple has long done well) on top of genuinely excellent performance and efficiency thanks to the progress with Apple Silicon. As far as program support is concerned, macOS actually comes out ahead of Windows on this front owing to a number of the software I use professionally actually being developed for Unix systems (macOS and Linux) and Windows either being an afterthought or wholly unsupported, with most other major software today being distributed across both Windows and macOS.

On top of this, I started researching the benefits of the Apple ecosystem—the synergies I’d benefit from by using both macOS and iOS together—and suddenly started feeling a bit of excitement surrounding the prospect of moving to macOS. As it turns out, there’s a bunch of cool functionality and conveniences to be gained. It also felt like a nice opportunity for a clean start of sorts, a chance to rethink my file management system from the ground up along with other facets of my digital life in terms of the tools I use and how I organize things. While a part of me is reluctant to be willingly handing over some degree of my digital sovereignty to Apple by investing in their ecosystem, I would argue that it is nigh impossible for most of us to fully escape the clutches of the big tech companies today, so I might as well choose the company whose values best align with my own and that maximizes convenience. (No big tech company is perfect; see Wikipedia’s nicely compiled list of criticisms of both Apple and Microsoft, along with the others with pages in the “Criticism of companies” category.)

So, I pulled the trigger and bought an M4 MacBook Pro, and I am happy to report I have absolutely no regrets two and a half months in! I have genuinely had a great time on macOS, despite the current version 26 Tahoe being controversial among the established Mac community. It has not been without its growing pains. There have been plenty of moments where I have to search for how to do something or have been a bit frustrated at something I’m used to doing quickly in Windows being a bit more arduous in macOS. Often though, I will find there is a sensible “Mac-native” approach to accomplishing my goal that is simply just different from how Windows had trained me to think, and this time I am embracing these differences with an open mind instead of faulting macOS for not being Windows. I will also add that the Supercharge app has been worth every cent I paid for it and addresses many of my remaining grievances with macOS so far.

Furthermore, (re)learning all of the new keyboard shortcuts has been an ongoing challenge, though with time I am internalizing patterns for how things work and realizing there is some degree of logic to when the various modifier keys are used. On that note, I am very happy that my beloved em and en dashes are typed far more sensibly on macOS (Shift + Option + hyphen / Option + hyphen) than the Alt codes on Windows, and it gives me a little bit of pleasure each time I discover yet another symbol on macOS that is sensibly typed rather than requiring memorization of a four-digit code. Developing and enacting my revisions to my file management and overall digital strategies has been a substantial amount of work, though I am very happy with how organized my previous mess of files now is. (This is more the fruits of my hard labor, not really a macOS benefit though.)

I have also enjoyed the passionate online media and community around Apple and macOS, with podcasts helping me fulfill my power-user nature on macOS and casually following news and rumors around Apple, news sites where I can keep up-to-date with new features being added to macOS and iOS and future releases, and blogs that have chronicled Apple for many years. I feel like I have learned quite quickly and have already achieved some macOS power-user-level feats in making tools for my personal workflows and resolving minor annoyances.

I must admit, I have felt a degree of schadenfreude in seeing the news covering the mess that has been buggy Windows 11 updates in early 2026 wreaking havoc, like I’m witnessing the results of AI slop code being pushed out to the masses recklessly without adequate testing. I understand that Apple is widely regarded as having fallen way behind on AI, but this is a feature, not a bug, for me. I don’t know why Apple has failed to achieve their initial AI goals, but I like to think it isn’t because they lacked the talent but at least partially because they are holding themselves to a higher standard of quality and reliability than the rest of the tech industry clambering to “move fast and break things” in the pursuit of AI rollout. Perhaps they underestimated the difficulty of developing an AI that can actually be trusted to have permissions on your phone or computer, but I am grateful Apple has chosen to withhold AI features and take a bit of egg to the face instead of following this trend of shoving unhelpful (and sometimes destructive) AI everywhere. I just hope this remains to be the case moving forward.

I do not see myself becoming one of the Apple fan-boys my past self would have scoffed at—I have no intention of blindly defending any poor decision-making at Apple—but man am I thrilled about actually enjoying my personal computing experience once again.

Choosing city transit tickets in Bergen, Norway

2025-01-12T00:00:00-08:00

Bergen, Norway is an absolutely lovely city I have fallen in love with and am lucky to call home, and getting around it is made quite easy with its excellent public transportation network of busses, light rail, and boats, all operated by Skyss. A valid Skyss ticket is required for and will work on all of these modes of transport (no need for different tickets for each).

Full details on the boarding process are outlined here, but, in short, within the Bergen area you do not need to present/validate/scan your ticket to board transit but should be prepared to present your valid ticket to an inspector in the event of a ticket inspection, where you will face a fine if without a valid ticket. (Outside of the Bergen area, you must present your ticket when boarding.) Skyss tickets can be purchased in a number of ways, though using the Skyss Billet app is the most common and convenient; note that tickets are activated at the moment of purchase, starting the timer on their validity period. Skyss also provides a separate travel/route planning app Skyss Reise, which displays information about any expected delays or route changes owing to tunnel closures, extreme weather, traffic, or anything else.

Skyss operates throughout Vestland, the Norwegian county containing Bergen, and has divided it into a number of price zones, where Zone A contains all of Bergen and a large area around it—including the airport and its major nearby islands Askøy and Sotra (among others). Skyss sells tickets encompassing 1, 2, 3, or 4+ zones; however, most people visiting Bergen and using public transit are likely to remain within just Zone A. Thus, this article only covers single-zone ticket prices. Furthermore, the contents of this article reflect the Skyss policies and prices as of the time of writing (12 January 2025), which may have changed in time.

Skyss sells the following types of tickets (current ticket prices):

Single ticket (90-minute validity): †47 kr
24-hour ticket: 115 kr
7-day season ticket: 255 kr
30-day season ticket: 795 kr
180-day season ticket: 3975 kr

Note that the season tickets last exactly their stated duration (e.g., the 7-day ticket is valid for 168 hours from its moment of purchase). Here are some Google search links for converting various currencies to Norwegian kroner (kr / NOK): USD, GBP, EUR, CAD, AUD, NZD, JPY.

†The single ticket pricing is complicated by the travel discount system, which decreases the cost of each additional single ticket purchased within the last 30 days, to a maximum discount of 40%.

While circumstances usually make it easy to predict whether a 24-hour ticket is more sensible than single tickets, over longer periods it can be less obvious whether it makes more sense to buy single tickets, one or more 7-day season tickets (or “passes”, for brevity), or a 30-day pass. Thus, I have done the math here.

The below plot shows horizontal lines indicating the prices of the 30-day pass and one to four 7-day passes, and its curves show the cumulative cost of purchasing some number of single tickets “N”. The darker black curve assumes all single ticket purchases are made within a 30-day window and that the first single ticket is bought at full price (which should be the case for most visiting tourists); meanwhile, the lighter gray curve assumes all single ticket purchases are made with the maximum 40% discount (representing the theoretical minimum cost of that number of single tickets).

From this, we see that the pricing breakpoints making passes cheaper than single tickets—for most visitors without any pre-existing travel discount accumulated—as a function of the number of “rides” (uses of public transit more than 90 minutes apart, thus requiring separate single ticket purchases) are as follows:

For 6 or more rides, the 7-day pass (255 kr) is cheaper.
For 13 or more rides, two 7-day passes (510 kr) are cheaper.
For 21 or more rides, a 30-day pass (795 kr) is cheaper.

For visitors intending to make frequent use of the public transportation, buying passes (season tickets) covering the duration of their stay is very likely to be the most economical (and convenient) option.

For locals and frequent visitors to Bergen buying the single tickets regularly who do have some travel discount already accumulated, these breakpoints will be a bit different. However, this plot is hopefully still a helpful tool in determining under what circumstances each combination of tickets/passes makes the most sense.

While this article does not seek to be a comprehensive guide to transportation in Bergen, the dedicated airport express bus service Flybussen should be mentioned. It is not operated by Skyss (though appears in the Skyss Reise app travel route suggestions) and requires purchase of its own tickets though Flybussen, either in advance online or at the bus (slightly more expensive). If needing to get to the Bergen airport very early in the morning (e.g., for a 6 am flight) or if arriving at the airport late at night (particularly on weekdays) and needing to get into the city, Flybussen may be your only option aside from taxis/Uber, which are considerably more expensive. More information on the travel times and stops serviced by Flybussen can be found here.

What happened to the red Froot Loop?

2024-07-24T00:00:00-07:00

I recently went to the grocery store and picked up a couple of boxes of Kellogg’s Froot Loops cereal. Upon getting home I realized the two boxes were not the same: Only one contained red loops!

The two boxes are pictured above; the left one with red loops had a manufacture date of 4 March 2024 while the right box without the red loops was produced on 9 May 2024. After inspecting their ingredients lists, the only difference was that the older box contained radish within its fruit and vegetable concentrates while the newer one did not.

I was a bit curious as to why Froot Loops had become 33% less colorful, especially given that I grew up in the US where the Froot Loops have six vibrant colors using artifical food colorings, contrasted against the European version’s (until now) three natural colors—red, yellow, and purple. Unable to find any information online about this change (and noticing that even on the official Kellogg’s European websites that the images of the Froot Loops boxes had not yet reflected this change), I reached out to the Kellogg’s UK contact page to lightheartedly ask about the reason behind this change and whether it was a permanent one. Their excellent consumer affairs department got back to me promptly with the following explanation:

Thank you for getting in touch with us regarding the new Kellogg’s Froot Loops recipe.

From time to time we evolve our recipes so that we can continue to meet our consumers’ needs and expectations. We identified that the red vegetable colouring we use for the red loop in Kellogg’s Froot Loops has increased significantly in price as a raw material so in order to minimise any pricing impact for our loyal Froot Loops fans, we made the decision to remove this colour. Froot Loops will now be a feast of our yellow and purple coloured loops and most importantly, keep the great taste our consumers know and love.

Furthermore, to answer your question about whether this is a permanent change: yes, this is indeed a permanent change for Froot Loops from Kellogg’s.

So, there we have it; the mystery is solved. The red loops are permanently gone owing to a raw materials price increase. While this is slightly dissapointing news, I am happy to report the new two-toned Froot Loops are indeed just as delicious as their tri-toned predecessor. :)

Schengen schedule calculator

2024-02-07T00:00:00-08:00

If you have traveled to (nearly) any European country, you have probably entered the Schengen Area (pictured above), a group of countries that have abolished all border controls at their shared borders and, as far as freedom of movement is concerned, work kind of like one large country. However, unless you’re a citizen of a Schengen country or in a Schengen country in which you hold a residence permit, your stay in Schengen is limited in a somewhat unique way: You can stay in the Schengen Area for a maximum of 90 days in any 180-day period. More comprehensive information can be found here regarding the specific visa agreements Schengen holds with all non-Schengen countries and whether you can enter Schengen visa-free.

In short, this 180-day period is a rolling window, looking 180 days backwards from any given date and counting the days spent inside the Schengen Area in that window. The days in which you enter and exit Schengen (travel days) are also counted as days spent in Schengen. If you are a frequent Schengen visitor (backpacker, digital nomad, lover to a Schengen resident, travel addicted retiree, international man of mystery, etc.), this can pose a rather complicated scheduling problem to ensure you don’t overstay your legal welcome. Our calendar of uneven months does not make this any easier, and complications abound the moment you’re trying to schedule numerous short trips instead of monolithic nearly 90-day ones.

There are already nice Schengen calculators online, but I felt they all suffered some shortcomings that made them difficult and/or clunky to use for actual planning (but perfectly fine for checking the legality of an existing plan with fixed dates). I wanted a tool that could help me brainstorm potential plans, giving me feedback on the implications of each stay on any future, yet unplanned, trips into Schengen.

While this feels like the perfect task for a program, I wanted to make something more accessible. After all, a majority of the people wanting to perform these calculations probably don’t have Python installed on their computers nor are comfortable with running scripts in a terminal. And these are not very user-friendly ways of inputting data anyways. You know what is widely familiar, visual, interactive, and can store data? Spreadsheets!

So that’s exactly what I’ve made. Below I have linked to my Schengen Schedule Calculator spreadsheet and included a screenshot (hover to enlarge) of its input and results area, with disucssion of its usage following.

[Download link for Schengen_schedule_calculator.xlsx]

^{As an obligitory disclaimer, you are solely responsible for ensuring you obey all laws pertaining to visiting and staying in the Schengen Area. While a lot of attention and troubleshooting went into this spreadsheet, its results are not guaranteed. This site’s owner disclaims all responsibly for anything that may or may not happen to you. Before going through with any travel plan, make sure to double-check the travel dates!}

Note: The spreadsheet utilizes the XLOOKUP() function, which is only available in Excel 2021 and newer. So, opening the spreadsheet in older versions of Excel will result in #NAME? errors.

The spreadsheet is quite simple to operate. The cells in which information should be inputted are filled in a light blue color and have a bold black border around them. Principally, these are just the dates of tentative stays in Schengen. On the right side of the results area is a section specifying when the next possible entry date is for a stay of some specific desired length, and the first of these columns has a value that can be changed to whatever length of stay you want.

The results columns to the right of the input dates are self explanitory, specifying how long the stay is (including the travel dates), how many “spare Schengen days” one has as of the start and end dates, a “Yes/No” on whether the planned stay is illegal, a “Yes/No” on whether you’re “locked out” of Schengen after the stay (meaning you must wait some minimum number of days before being able to re-enter), and the next date on which you can legally re-enter Schengen and for how long at most.

In particular, column F reflects spare days at the start of the stay, and column G reflects how many days the stay could still be extended at its end (or, if negative, how many days must be removed from the end of the stay). A stay is marked illegal if either the start date is too early or the end date is too late.

Everything below this main input/results box (everything below row 16) is solely for the calculations being performed and should not be tampered with (and is quite complicated). If giving this spreadsheet to a friend or loved one that may be prone to inputting values outside of the designated cells and breaking the spreadsheet, I would recommend using Excel’s protection features to lock down all but the blue input cells of the spreadsheet before distributing it to others. In fact, here is a separate version of the spreadsheet where I have gone ahead and locked down all but the intended input cells:

[Download link for the "protected" version: Schengen_schedule_calculator_protected.xlsx]

I hope this spreadsheet can be a useful tool for anyone trying to plan out and schedule days in the Schengen Area, for themselves, a loved one, friends, or family!

To address a potential edge case, the sheet as currently designed—optimized for performance—is only capable of handling 10 “stays” and a maximum of 907 days between the first day of the month (minus one day) of the start date of the first stay (cell B22) and the end of the tenth stay (cell B928). Provided the substantial number of calculations being conducted within the sheet, expanding this much further can cause performance issues. However, if you wish to make plans spanning longer than 907 days and/or more than 10 stays, greater capicity is needed. Solving this issue, an “extended” version of the spreadsheet is linked below.

[Download link for the "extended" version: Schengen_schedule_calculator_extended.xlsx]

In this extended spreadsheet, “Sheet1” remains the same; however, the additional “Sheet2” and “Sheet3” are essentially extensions of the first sheet. Sheet2’s first “stay” will be a copy of the final provided stay on Sheet1, and the calculations further down in the sheet (rows 18 onward) will begin 180 days prior to the end date of the final stay provided on Sheet1, just copying over the relevant state of calculations for those first 180 days from Sheet1 (and thus preserving the dates relevant to the rolling Schengen stay limit). Sheet3 is the same but starting from the last stay provided on Sheet2. Note that this means the ordering of the sheets matters given that they reference each other! The sheets should be ordered chronologically; you can essentially think of each additional sheet as an extension of the previous one. If you wish to extend the number of sheets further than the three included in this spreadsheet, follow the below steps:

Right click the rightmost sheet tab at the bottom and select “Move or Copy…”
In the popup menu, tick the “Create a copy” box and select “(move to end)” then click “OK”
- Note that the copying process will take some time; you can monitor its progress on the right side of the bottom bar reading “Calculating (X Threads): ##%”.
In the newly created sheet, delete the contents of the cells in range C7:D15 (all light-blue shaded start/end dates in the table, excluding the first pair of dates with a dark cell color)
Rename the newly created sheet to be in sequence with the others (e.g., “Sheet4” instead of “Sheet3 (2)”)
Use the new sheet as normal!

The only caveat to mention is that this 907-day limit per sheet still exists, and if this is to be exceeded before the 10/9-stay limit per sheet, the next sheet will work fine but only if the final stay of the preceding sheet still fits within the 907-day limit (i.e., there should not be any errors in the main input/results table area for the last stay in any given sheet).

Controlling iCloud/OneDrive file sync status in CMD

2024-01-27T00:00:00-08:00

If you ever find yourself needing to control the sync status of a OneDrive or iCloud for Windows file from a batch file or CMD, it is fortunately quite straightforward with ATTRIB.exe built into Windows (comprehensive documentation here), which is used to view and change file attributes. Here, we are only concerned with three particular attributes (with their attrib flags in parenthesis): Offline (O), Pinned (P), and Unpinned (U).

In CMD, using the attrib command with just a filename (e.g., attrib filename.txt) without flags prints the file’s current attributes and path. By adding flags with a plus/minus (+/-) between attrib and the filename, the file’s individual attributes are changed and toggled on/off (True/False). Putting a plus before the flag tells it to set that attribute to True (e.g., +P) while a minus sign sets it to false (e.g., -P). Each of the various icons indicating sync status of a file corresponds to some combination of these flags being enabled or disabled:

Attributes	Set with	Explanation
`O U` / `O`	`+U`	“Only available online” files, in the cloud only and not locally available
`P`	`+P`	“Always keep on this device” files, locally available and will be kept downloaded
` `	`-P` / `+P` then `-P`	“Available on this device” files, locally available (will be made online-only if space is needed)
`O P` / `O`	n/a	Sync in progress (status when a file is downloading or queued to be downloaded)

Note that the Offline (O) flag is something that just updates automatically and is not something we set. Setting U or P to True automatically sets the other to False, resulting in Unpinned (U) files being online only and Pinned (P) files being always locally available. Both U and P can be set to False simultaneously, resulting in the file being locally available but not necessarily always available.

In the picture below is an example with a file called IMG_6736.MOV in my iCloud Photos library. The file goes through the following steps:

It begins as online only (O)
It’s then “pinned” (+P) to be always kept on this device
We check to see the attributes while queued/downloading (O P)
We check to see the attributes after the download finishes (P)
Its “pinned” status is revoked (-P) to remain available but not necessarily always downloaded
We check its attributes and see both U and P are False
It’s finally “unpinned” (+U) to remove the download and make it online only again
And we check its attributes to verify its online only status (O U)

The iCloud Photos use case

My motivation for looking into all of this in the first place revolved around iCloud Photos and its abhorrently terrible performance when trying to download many files at once. This seems like a normal use case, no? I have just downloaded iCloud for Windows, and I want local copies of all of my iPhone’s photos that are synced to iCloud. I have nearly 28,000 photos, so I right click the “Photos” folder and select “Always keep on this device” then go to bed. I wake up to see in the window from the iCould taskbar tray icon that there was essentially no progress in eight hours. After some troubleshooting, I eventually right click the “Photos” folder and select “Free up space” to stop this stalled process.

If iCloud can’t handle trying to download 28,000 files, how many can it? After some amount of testing, I discover 50 is about the sweet spot. If I select 50 files and set them to “Always keep on this device”, it takes on the order of five minutes (to a bit less) before they are all downloaded. If I do the same but with a selection of 75 files, this time increass to a bit over 10 minutes. With 200 files, it varied but was on the order of an hour. When I had 500 selected, it took well over six hours. It seems iCloud gets increasingly bogged down the longer its queue for files to be downloaded is.

So, what am I to do? Am I really going to babysit my iCloud Photos File Explorer window and spoonfeed it 50 files at a time to download every five minutes or so… for over 550 cycles?!? Assuming perfect efficiency, that’s 45 hours of tending to it. Arguably better (but perhaps wrose even), I could feed it ~500 files to download overnight and do this nightly for nearly two months… These are obviously not sensible solutions.

What I needed was a way to automate this. A scripted solution. Thus, the batch script at the bottom of this article was created. The script scans through all files in the “Photos” directory and sets attrib +P to any files it sees are currently set as Offline (O). Once it has “pinned” 50 files, a sleep/timeout timer of 300 seconds is activated to give iCloud time to download the files. The counter is then reset, and the scan continues.

One small hiccup I did encounter was that the substantially larger (and sporadically located) .MOV video files were taking longer to download and would sometimes cause a backup where the 5 minute timer was insufficient to download the 50 flagged files if they contained too many/large .MOV files. This then spiralled the queue out of control as iCloud choked on the increasing number of files being fed to it, slowing down even more the further behind it got. To circumvent this, I would suggest running the script first with the directory scan only looking for .MOV files (for %%f in (Photos\*.MOV)) with a lower counter value before the sleep timer is activated, such as if !count!==10, and perhaps with a longer sleep timer too depending on your video file sizes and network speed. After all the larger video files are downlaoded, just rerun the script but searching for all file extensions and allowing the count to get up to 50. The counter limit and sleep timer duration are values you can play around with and optimize for what prevents backlogs in your own environment.

As a warning, it may be tempting to reduce the sleep timer as much as possible to maximize efficiency, the percentage of time spent actually downloading files rather than sleeping. However, especially if you would plan to leave the script running unattended for hours, I would caution against getting too close to “maximum” efficiency. Remember, when iCloud gets overburdened with too many files queued, it slows down, meaning if you keep feeding it more files to download at a constant rate, it will just continuously slow down more and more and fail catastrophically, getting hung up and (nearly) ceasing all downloads. Thus, I think it is best to give it ample time to download the “slowest” batch of files fed to it at a time.

The below Windows batch script was stored in and executed from the “iCloud Photos” directory, which contains the “Photos” subdirectory holding all of the photo and video files synced to iCloud Photos. Beneath it is some more details on the script.

@echo off
setlocal enableextensions enabledelayedexpansion
set /a count = 0
for %%f in (Photos\*.*) do (
  attrib %%f
  attrib %%f | findstr "^.......O" >nul && (
    attrib +P %%f
    set /a count += 1
    if !count!==50 (
      echo Current time is !time!
      echo Waiting 5 min for previous 50 files to download...
      set /a count = 0
      timeout -t 300 >nul
    )
  )
)

The script’s first for loop scans over all files in the Photos directory. The attrib command is called on its own just to print to the terminal all the files and their attributes as the code progresses. The next attrib call has its output piped to a findstr call that looks for an “O” in the eighth position; for a file with the Offline attribute, the eigth character in the string printed by the attrib call is an “O”, hence the seven periods followed by an “O” in the findstr call. If an “O” is found, the code executes attrib +P to the file and increments the counter variable count for the queued files. The loop proceeds like this until the counter reaches 50. Then, the code prints the current time and waits for 300 seconds, set by the timeout call, to give iCloud time to download the 50 files that had just been pinned and slated for download. The count variable is reset to 0, and then the code proceeds with this loop until all files have been cycled through.

My initial version of the script I came up with is shown below, with its own explanation following. It is similar but is quite a bit slower at scanning through the directory.

@echo off
setlocal enableextensions enabledelayedexpansion
set /a count = 0
for %%f in (Photos\*.*) do (
  for /f "delims=" %%a in ('attrib %%f') do set "readvalue=%%a"
  echo !readvalue!
  set IsOffline=0
  call :CheckAttributes
  if !IsOffline!==1 (
    attrib +P %%f
    set /a count += 1
    echo !count! files in download queue
    timeout -t 1 >nul
  )
  if !count!==50 (
    echo Current time is !time!
    echo Waiting 5 min for previous 50 files to download...
    set /a count = 0
    timeout -t 300 >nul
  )
)
:CheckAttributes
  set /a ichar = 0
  for /F "delims=" %%G in ('cmd /D /U /C  echo !readvalue!^| find /V ""') do (
    set /a ichar += 1
    if "%%G"=="O" (
      set IsOffline=1
      goto :break
    )
    if !ichar!==17 goto :break
  )
:break

The script’s first for loop scans over all files in the Photos directory. The line immediately beneath that is essentially just executing attrib filename.txt but storing the string output of that command into the variable readvalue. IsOffline is initialized as 0, and then the CheckAttributes function (defined at the bottom, outside the main loop) is called, which scans readvalue character by character, checking if any of the characters are "O" (if "%%G"=="O") indicating the presence of the Offline flag and setting IsOffline=1 if an “O” is present. To avoid reading characters in the file path portion of the attrib output, the function terminates and returns to the code’s main loop after checking the first 17 characters (or once an “O” is found).

Then, if IsOffline is 1, the code executes attrib +P to the file and increments the counter variable count for the queued files. The loop proceeds like this until the counter reaches 50. Then, the code prints the current time and waits for 300 seconds, set by the timeout call, to give iCloud time to download the 50 files that had just been pinned and slated for download. The count variable is reset to 0, and then the code proceeds with this loop until all files have been cycled through.

The slowest part of this code is the character-by-character check for the attributes in the CheckAttributes function call. Perhaps there is a faster way, but in my troubleshooting I had difficulty getting other approaches to work. And, as I stated earlier, efficiency wasn’t my greatest concern given how slow iCloud is in the first place.

Also, while my script searches for the presence of the Offline flag, a slightly more robust way would be to search for the absence of the Pinned flag. In the script, renaming IsOffline to IsPinned and changing the logic in the first if statement to be if !IsPinned!==0 would accomplish this. I did not implent it this way myself because this essentially made the default action of the code to “do something” if my attribute checking were to fail in some way, though I now believe it would be perfectly fine.

As a disclaimer, I only tested all of this on iCloud for Windows files; however, iCloud for Windows works in the same way as OneDrive for file syncing attributes. In fact, as I was troubleshooting all of this myself, all of the resources I used referenced OneDrive.

AI-generated artwork

2022-09-23T00:00:00-07:00

Inspired by a recent news story and an episode of the Cortex podcast, I made the below presentation to present to my friends and colleagues informally at a social meeting of the PhD students and postdocs in my department/institute at HVL.

[Direct link to PDF of the presentation]

This presentation was originally given at an HVL PhD social meeting on 23 September 2022.

Is nuclear power our savior?

2022-02-25T00:00:00-08:00

Nuclear energy is one of our best tools on the path to a greener future, triumphing where many renewable technologies struggle in reliability and output, while being similarly—or even more—green and safe. How is this so?

Finland’s Olkiluoto nuclear power plant’s newly opened unit 3 (left) and older units 1 and 2 (right). Source: Wikipedia

Energy is mankind’s most valuable resource, behind only water. Harnessing it propelled humanity into the industrial revolution, first converting thermal energy simply to mechanical energy and later, perhaps the greatest revolution, to electrical energy. Most global electrical power is still produced through these means: heating water to make steam to turn a turbine to generate electricity. Traditionally, this heat came from the combustion of fossil fuels; however, the past century has seen a few innovations on that front. The most substantial of these is nuclear energy, currently responsible for about 10% of all electricity produced globally.

Nuclear power exploits the nature’s strong nuclear fundamental force, letting us convert mass to energy per Einstein’s famous E=mc². The density of this energy is staggering. A 10 g uranium fuel pellet in a reactor releases approximately the same energy as 1.3 tons of coal, 1300 L of oil, or 1100 m³ of natural gas, about 40 gigajoules (~11,000 kWh_th). The “breeder” reactors of the future increase these numbers at least tenfold with greater utilization of the nuclear fuel while simultaneously dramatically reducing the radioactive longevity of the spent fuel left behind.

Nuclear plants also boast the highest uptime at maximum output of all energy sources, making them an extremely reliable source of baseload electricity (the constant demand, regardless of time of day), a role none of the other, often weather-dependent, low-carbon energy sources are well-suited for, aside from hydro in abundance, which Norway happens to be naturally blessed with. Nature did not deal such a generous hand to the rest of the world.

In the pursuit of a safer and carbon-free energy future, what role does nuclear play? Per GWh electricity produced, nuclear is actually a bit greener than wind or solar in greenhouse gas emissions and results in comparably few deaths from pollution and accidents as renewables. For reference, these emissions and fatality figures from oil and coal are hundreds of times larger. It seems evident renewable energy should join hands with nuclear to overcome some of its largest shortcomings in reliability and throughput.

Some challenges persist, more political than technical. Radiation’s invisibility and mystique easily trigger our evolutionary instincts to fear the unknown, and this is further amplified by nuclear energy’s unforgettable origins and few high-profile accidents, leading to a large “not in my backyard” sentiment. Risk assessment is not one of our natural strengths; many similarly fear flight but feel at ease in cars, despite the safety statistics painting a starkly different picture. Another challenge is economic. Though nuclear fuel is cheap, startup costs are high, and investors are too impatient to wait for easy profits decades away, especially if seen as a risk due to political volatility. Fortunately, as public knowledge on nuclear is rising, so is its image and its odds of helping us achieve a greener future.

Quick facts

Electricity produced in 1 year: 1 nuclear plant = 980 wind turbines, 25500 m² solar panels, or 100 average Norwegian hydroelectric plants.
Nuclear accounts for the least CO₂ per GWh produced of all major energy sources.
Deaths from accidents and pollution per GWh for nuclear are 341 times lower than for coal, 263 for oil, and 40 for natural gas.

This article was originally written to be translated to Norwegian and submitted to Bergens Tidene for publication; the text featured here is the original article pre-translation. A dialog-formatted version of this text in Norwegian (nynorsk) was published in the 19 May 2022 issue of Bergens Tidene.

Interactive world data map generation tool

2021-03-14T00:00:00-08:00

This page serves to discuss the usage and behaviors of the Interactive world data map generator tool that I wrote. The GitHub page’s README covers required packages and very basic usage, so this page seeks to take a deeper look at the intended input formats and settings, the logic behind some of the code’s options, and how the maps generated by this code differ from those generated by the base Pygal library which this code uses.

This code serves to easily create interactive world maps such as the one below showing the number of Olympic medals won by each country. The code’s input data can be in an Excel spreadsheet, a .csv (comma separated values) file, or a .tsv (tab separated values) file. (Any other file formats provided are assumed to be CSV files.) It outputs a Scalable Vector Graphics (SVG) file containing the map and an HTML file with the map embedded along with a table of all of the data on the map.

Hovering over a country reveals the value associated with it (if nonzero). Hovering over each bin in the legend will highlight all countries in that bin, and clicking the bins in the legend will toggle whether those countries belonging to each bin are shown on the map or not.

[view standalone SVG in fullscreen]

Basic operation

When the code is ran, the GUI shown below will pop up (but with default entries). This article will discuss these various settings.

The default value of “Input file” will be either a user-provided file path taken from the first line of a file titled “default_path_to_data_file.txt” if the code detects this .txt file in the working directory, or it will default to “default.xlsx” in the current working directory. A different file can be selected using the Browse button or by manually entering a new path. When the code is ran for the first time for a given input file (or if the input file has been updated since the last time the code was ran), the code will carefully parse the file and print diagnostic information to the screen, allowing you to check if the listed countries are being properly identified. If “Run” is selected, the window will close and this information is printed to the terminal as usual; if “Run (keep window open)” is selected, a popup window (shown below) to which the terminal output is redirected will appear.

Keeping the window open can be useful for trying different colors and settings quickly. After the file has been carefully parsed, its data and the GUI’s settings are saved to a file of the same name as the input data file but ending with the .pickle extension instead. Then, in future cases when the code is ran, if it discovers that this pickle file is present in the same folder as the source data file and that the source data file has not been modified since the pickle was created or last updated, the code will use the data stored in the pickle file instead of rereading the source data file, speeding the code up notably. Additionally, anytime the code is ran and the corresponding pickle file is discovered, the GUI will automatically change its settings to match those used the last time the pickle file was updated, and any new or modified settings will be saved to the pickle file for future iterations. For Excel spreadsheet files, you can select data from any of its sheets, and the pickle file will independently save the data from and GUI settings used for each sheet. Updating the Input file or the sheet name causes the code to search for the pickle file and to update the GUI if found.

Tallied vs Untallied data

The most important option provided by the code is the type of data: tallied or untallied. Tallied data is presented as a list of countries with values already assigned (population per country, Olympic medals won per country as shown at the top of this page, etc.) and would be in a spreadsheet/data file tabulated as shown below. In this case, the code needs to know the column numbers for both where the country names are listed and where the data values are listed. If a country is identified multiple times, the values from each are summed; for instance, in this example the number of medals for Germany includes those from “Germany”, “West Germany”, “East Germany”, and the “Unified Team of Germany”.

Country	Total medals won (Summer + Winter Games)
United States	2828
Russia (and Soviet Union)	1776
Germany (modern, East, West, and United Team of)	1754
United Kingdom (Great Britain)	883
France	840

Untallied data consists of just a list of countries, and the number of appearances of each country is counted. This would be useful if you hosted an international event, had a list of all of the attendees and their countries, and wanted to make a map illustrating how many attendees you had from each country. The code takes care of summing up the number of attendees per country for you. The example of this type of data presented below is generating a map of the number of Olympic Games (Summer and Winter) hosted by each country provided a list of the locations of all Olympic Games. For the sake of this example, canceled events are excluded while tentative future events are included. Some of the data is presented in the table below.

City	Country	Year
Atlanta	United States	1996
Nagano	Japan	1998
Sydney	Australia	2000
Salt Lake City	United States	2002
Athens	Greece	2004
Turin	Italy	2006
Beijing	China	2008
Vancouver	Canada	2010
London	United Kingdom	2012
Sochi	Russia	2014
Rio de Janeiro	Brazil	2016
Pyeongchang	South Korea	2018

To generate the map, the number of instances each country name appears in the data is counted. For example, the United States appears 2 times in the above table (but a total of 9 times in the Wikipedia table including past and future events). Once the data is tallied, the map below can be generated.

[view standalone SVG in fullscreen]

For both tallied and untallied data, the code also always needs to know how many header rows there are (rows at the top which will be skipped). And, as noted in the GUI, letters (as used by Excel) may be used for the column numbers. If Excel-style letters are provided for column numbers for non-Excel files, they will be converted to integer numbers.

The map of Olympic events per country above was generated with the following settings in the GUI:

Output options

The output folder will default to the same directory as the input file, though this can be customized. This is where the output .svg and .html files will be written, and their filenames can be customized.

The maps have a single base color. The countries in the highest bin will adopt this color while lower bins will have this color mixed with increasing amounts of white. Clicking the “Pick color” button opens a color picking window, and the color of the button afterward will be updated to reflect your selection.

The HTML embedding options for the SVG file are discussed in detail later on this page due to being a slightly more complicated topic.

Most of the text fields have self-evident names, controlling the map title, the text appearing in the boxes when hovering over a country on the map, the text in the legend in the bottom left of the map, and the column titles of the HTML table. UTF-8 characters are supported in these fields, not just ASCII.

Binning

The custom binning of data is one of the most valuable features of this tool, and a variety of options are available. In principle, there are four main binning styles: (1) providing manual bin edges, having the code automatically calculate evenly spaced bin edges on either a (2) linear or (3) logarithmic scale, or (4) using the default binning structure of Pygal, ignoring all of the other binning features (not recommended and only available when “Show legend” is disabled).

When manually entering bins, they must be provided as a list of increasing numbers separated by commas. Note that you are specifying bin edges here, meaning the nominal number of bins is one less than the number of values provided.

When automatic binning is selected, the code automatically constructs evenly spaced bins in either linear or logarithmic space, and you can specify the desired number of bins.

There are two special options which impact the meaning of each bin: (1) the overflow bin and (2) forcing integer bin edges. When the overflow bin is enabled, an extra bin containing everything above the original final bin maximum is added to the plot. In the case of the automatically generated bins, this effectively just increases the selected number of bins by 1.

Forcing integer bin edges does a few things. First, the automatically generated bins edges, which often are decimal values, are forced to be integers. (Manually entered bin edges are not affected.) Second, it causes the data to be interpreted as if it were “counted” integer data (regardless of whether it actually is). When this is the case, the left edge of each bin after the first one is increased by 1. When forced integer bin edges are disabled, the data is assumed to be decimal, and the bin edges are set to contain all numbers between the minimum and maximum listed value.

By default, both of these options, the additional overflow bin and forced integer bin edges, are enabled. The behavior of these two settings is illustrated in the table below where “V_lower” and “V_upper” are consecutive bin edges (either specified manually or automatically calculated) and where square brackets [] include the values next to them ([v1,v2] means ≥v1 and ≤v2) while parentheses () do not include the values next to them ((v1,v2) means >v1 and <v2).

Binning Logic
	add overflow bin = `True` Yes	add overflow bin = `False` No
force integer bin edges = `True` Yes	First bin: [V_lower,V_upper] Mid bins: [V_lower+1,V_upper] Last bin: [V_lower+1,V_upper] Overflow bin: [V_upper+1,∞)	First bin: [V_lower,V_upper] Mid bins: [V_lower+1,V_upper] Last bin: [V_lower+1,V_upper] Overflow bin: n/a
force integer bin edges = `False` No	First bin: [V_lower,V_upper] Mid bins: (V_lower,V_upper] Last bin: (V_lower,V_upper] Overflow bin: (V_upper,∞)	First bin: [V_lower,V_upper] Mid bins: (V_lower,V_upper] Last bin: (V_lower,V_upper] Overflow bin: n/a

As an example, the resulting bins from the four permutations of these special options for manually provided bins 1, 5, 10, 20, 50 are shown in the table below.

Special options settings	Produced bins
force integer bin edges = `True`, Yes add overflow bin = `True`, Yes	[1,5], [6,10], [11,20], [21,50], [51,∞)
force integer bin edges = `True`, Yes add overflow bin = `False`, No	[1,5], [6,10], [11,20], [21,50]
force integer bin edges = `False`, No add overflow bin = `True`, Yes	[1,5], (5,10], (10,20], (20,50], (50,∞)
force integer bin edges = `False`, No add overflow bin = `False`, No	[1,5], (5,10], (10,20], (20,50]

And as you may have noticed on the Olympic Games hosted map, when forced integer bin edges is enabled, any bin with width 1 and containing only a single value is printed to the legend as that single value rather than as a range.

HTML embedding options

Several different options are provided for embedding the SVG file into HTML since different web backends may behave differently with each method and only allow some to work. While <object> is generally recommended by online resources, <embed> is used for the maps shown on this page. It is generally not suggested to use <iframe>, but it is included for completeness. All three of these nominally let the SVG file behave like normal, retaining all of its interactivity. When using <img> this interactivity is completely destroyed, and the SVG will just be rendered as a static image. (Note that none of these options affect the SVG file, just how it appears in the HTML page.)

By default, the SVG files look fine in the unstyled HTML file produced or on their own. However, depending on your web backend, you may also need to make some minor modifications to the HTML code and the SVG file itself for it to render optimally (in terms of the gray border’s size and positioning) on your own webpage. For the SVG files shown on this page, I opened the SVG file in a text editor (like Notepad++), and on line 2 within the <svg> tag I replaced viewBox="0 0 800 600" with viewBox="0 100 800 400" preserveAspectRatio="xMidYMid meet" width="100%" height="100%" to get the positioning and framing just how I wanted it. When inserting the Olympic events map from earlier into this page, I used the below HTML code:

<div class="fluid-width-video-wrapper" style="padding-top: 75%; text-align: center;">
<embed src="/files/world-map-tool/Olympic_events_per_country.svg" type="">
</div>

Again, depending on your own backend and styling (CSS and/or <style> tags), you may need different adjustments to make the maps appear just right on your own webpages.

How do these maps differ from those made with stock Pygal?

The Pygal library is used by this tool for generating the basic core of these maps. The map generation capability is just one of a variety of very handy functions of the Pygal library. However, there is quite a bit of value added by using this tool versus Pygal on its own.

Below is the same map as presented at the top of this page but without any of the additional processing performed by this tool aside from the parsing of the data spreadsheet to automatically populate the dictionary object (with ISO2 country names as the keys) that Pygal accepts as its input, something you would need to construct manually if using Pygal alone.

[view standalone SVG in fullscreen]

So, in addition to handling the rather time-consuming task of manually constructing the input dictionary object, using Pygal on its own for constructing world maps can produce rather lacking results. There is no way to control the binning of the data, meaning for some datasets the maps are not particularly informative to look at. Additionally, there is no way to display the binning used by Pygal, meaning the maps feel subjective in nature as well, and the text appearing in the legend is identical to the next in the hover boxes. You may also notice that some of the country names used by Pygal are either uncommon, unnatural, or slightly controversial.

This tool provides a GUI for easily generating these maps, an automated way to parse and tally data (a nontrivial task in regards to translating country names in a large variety of formats to ISO2 codes, thanks to coco), the ability to fully customize the data binning structure, the ability to customize every visible descriptive text field independently, and presenting the country names in more common formats (again, thanks to coco).