Nelson Chen's Blog

My wobbly VHS pipeline

Sun, 13 Sep 2026 10:34:51 -0700

I wanted to recover some VHS tapes. Somehow this turned into tapping a signal inside a Sony VCR, putting capture cards in a computer from the junk pile at work, spreading the processing across three machines, and asking Codex to help redesign an amplifier PCB.

There is a reason I went this way. The usual way to digitize VHS is to connect a VCR’s video and audio outputs to a capture device and record them on a computer. The VCR reads the tape and turns its signal into a picture before the computer ever sees it.

For better results, you can look for a nicer VCR with a built-in time base corrector, or TBC, which helps correct playback timing errors that make the picture wobble. Unfortunately, the decks I was looking at were $$$. With ancient media having a moment again, it feels like everybody wants the same aging hardware. The expensive stuff is, well, expensive.

VHS-Decode offers a more hardcore route. You still need a working VCR, but you tap the RF signal inside it, near the tape-head amplifier, before the machine turns it into ordinary video. This is not the antenna connector on the back. Suitable capture hardware records that earlier signal, and the computer reconstructs the picture, including time base correction, in software.

That sounded pretty appealing: use an ordinary VCR, spend less on a fancy deck, and accept more fiddling. Keeping the RF recording also means I can try different settings or a better decoder later without playing the physical tape again.

I first heard about VHS-Decode about two years ago. I lacked the hardware, time, and initiative to really start. Codex helped me get past a lot of those hurdles. I drove the whole setup through Codex, right down to configuring Ubuntu on the capture machine. It was involved from the beginning, not something I added after getting everything working.

To be clear about who wrote what: Codex wrote a lot of the custom code for this setup. I directed and oversaw the work, made the decisions, and handled the physical hardware. When I call these “my scripts” or “my pipeline,” I mean the setup I am building and responsible for, not that I personally typed all the code.

The pipeline repo has an AGENTS.md with instructions for Codex and a journal.md where we take notes as we go. That journal now runs through months of experiments, failures, and procedures that apparently needed another procedure.

It is still a constant WIP. But looking back through those notes, there is an actual journey from “can I get a signal?” to something that helps me understand and organize the tapes too.

First, get a picture

I already had the Sony SLV-688HF. The capture computer was a Dell OptiPlex 7010 rescued from the junk pile at work. An old business desktop with room for capture hardware seemed like a reasonable place to start.

In late June and early July, the work was mostly getting that machine ready: Ubuntu installation and configuration, the CXADC driver, capture cards, cables, and figuring out whether the signal coming out of the VCR was useful. I was directing that work through Codex while handling the physical hardware myself.

On July 3, the first input we tried was basically flat, even with the tape playing. Switching from the white BNC input to the red one gave us a much healthier signal. A small detail, but fairly important when the whole project depends on recording something other than nothing.

That day’s one-minute test produced about 1.6 GiB of RF data. Decoding it on my MacBook took a little over five minutes in that early test, then produced roughly another 3.2 GiB of intermediate picture data. This was already a hint about where the project was going.

First there was a viewable black-and-white preview, without sound. Then a short color preview.

There was an actual picture. The weird route worked well enough to keep going.

One minute was manageable. Two hours was different.

The next problem was doing this at tape length. I added a Samsung 980 1 TB NVMe SSD I already had on hand as scratch storage in the capture machine. By August 8, a two-hour-and-two-minute capture had produced about 196 GiB of RF data, plus separately recorded audio. The machine’s 250 GB boot SSD was not going to provide comfortable working room for that.

Now storage and transfers were part of the job. The source needed to get onto the storage machine, and the copy needed checking before I treated it as safely transferred. Then came decoding, with another set of enormous files to put somewhere.

I store the captured RF on the NAS in FLAC. Yes, FLAC, the format usually associated with lossless audio. Here it holds the sampled RF signal, not a finished video. The separately captured audio has its own FLAC file. This saves space without throwing away the signal samples I want to decode again later.

I had Codex write custom scripts and capture tooling, under my direction, to FLAC the streams on the fly. That avoids making every capture a huge raw file followed by another whole compression job. The compressed captures still need to get safely onto the NAS, and the tooling has to keep track of whether samples were lost or an encoder failed. Compression is useful; silently losing part of the tape is not.

I tried decoding on the storage machine too. One run was heading toward roughly two days of processing. Later the production decoding moved to a Windows machine. The setup grew into three main jobs on three machines: capture, preservation storage, and decoding/export.

The Windows stage now turns the retained RF recording into intermediate luma and chroma picture files, then makes H.265/AAC copies I can watch. Those convenient video files do not replace the preservation source. Keeping the earlier signal is a large part of why I chose this approach in the first place.

Audio added its own little complication. I capture it separately, and separate devices have separate clocks. Their crystal oscillators can run at slightly different rates, so audio that lines up at the start can drift away from the picture later.

Synchronizing the hardware clocks is one approach. I was okay with resampling the audio to fit the video timeline instead. That means adjusting its timing over the recording, not just moving the whole track left or right and hoping. It also means checking synchronization beyond the beginning.

At this point, “capture a tape” had acquired quite a few more steps than pressing record.

Then I had to find out what was on it

An August entry in that journal has a useful lesson in why the end of a recording is not necessarily the end of a tape.

A fixed-duration capture can keep recording after a short tape has stopped, wasting a ridiculous amount of space. But looking for static is not enough to decide where useful material ends either. One tape had a static gap followed by another recording. Stopping at the gap would have missed it.

These tapes were recorded over, sometimes only partially. A cassette can contain television, home video, and then more television. Sampling a few places and finding TV does not establish that there is nothing personal elsewhere.

So my rule is to capture the whole tape first and analyze it afterward. I do not stop because the interesting-looking recording ended, television started, or the picture went to static. There could be something else further along. The power meter helps detect the physical end of the tape; figuring out which parts to keep in the viewing copies is a separate, later job.

This became one of the most useful parts of having Codex involved. It has been great at scene analysis, helping with transcription and translation, and writing up what is happening on the tapes. It can help sift the home-video sections out of recorded TV programming and leave a map of the recording alongside the watchable files.

The review procedures grew around that work: frames throughout the recording, extra scene-change samples, and moving picture with audio around anything ambiguous. I want to know what was actually checked, not just get a confident summary based on three screenshots.

Codex also helped read the wonky Chinese handwriting on the old labels. Some of it is barely readable scratches at this point. A plausible reading and translation can give me a starting point, and the footage can help make sense of it. Uncertain characters need to stay marked as guesses. The same goes for difficult speech in a transcript.

I am keeping the tape contents and detailed records private, but this part matters to the story. Getting a video file is useful. Having help figure out what is in it, where the different recordings start, and how to find them again makes the whole project much more useful to me.

Of course it grew a labeling system

By mid-August, I had also connected a Brother PT-D600 label printer over USB. Apparently the tape project needed a labeling regime too.

I stopped using printed status labels. Keeping those updated was more fuss than it was worth; off-the-shelf smiley-face stickers from Amazon were easier.

The labels I still print are for identity and content only. Each tape gets a stable ID, and its box gets the same one. Content labels describe the recording, with a date when known. Processing status stays in the digital records, not on a succession of replacement labels.

Yellow means “could be better.” Green is reserved for what I would call a perfect capture. I am not there yet: I still do not have the shared-clock hardware to synchronize the captures, so I am using the resampling approach described earlier. Green is a goal for now, not a claim about the captures I already have.

That connects the digital notes back to the object in front of me. Codex can help inspect a tape, write up the findings, and print the labels. If the box and cassette get separated, I can still identify both.

There is also an M5Stack Atom Lite, a little ESP32 device with a built-in infrared transmitter, for sending the VCR commands like play, stop, and rewind. It cost me $14. I 3D-printed a small stand for cents to hold it up against the VCR, aimed at its infrared receiver. So the automation can press the remote buttons too, although sending an IR command is not proof the VCR obeyed it.

For that, I also monitor the VCR with a power meter. The deck draws different amounts of power when the tape transport is active and when it is stopped with a tape loaded. The workflow watches those measured states rather than assuming an IR command worked. To rewind, it sends Stop, checks that the deck has settled, sends Rewind, then watches for sustained activity followed by a stable return to the stopped state. Those checks are specific to this VCR and setup.

At capture time, the software starts recording before sending Play, so it does not miss the beginning while getting ready. It then watches for the deck to become active. When the tape reaches its physical end and the transport stops, the power draw settles back into the stopped range. That sustained change is how the workflow detects end-of-tape, alongside the capture’s own integrity checks. It records through the TV programming, static, and silence instead of using the content to guess where the tape ends. An unexpected stop or ambiguous meter evidence still needs checking; a power reading alone cannot explain why the transport stopped.

Then, later in August, I had it help redesign the RF amplifier PCB for a more ordinary, externally regulated 9V input. I wanted the power setup to be less exotic. This was a later hardware experiment, not something I had to finish before getting that first picture in July.

The redesign passed KiCad electrical and layout checks, and the prototype fabrication order shipped. That is not a bench-tested result yet. Still, going from a project I had not managed to start to ordering a revised amplifier board is quite a progression.

The setup kept finding new ways to be annoying

Moving the decoding to Windows did not end the infrastructure work. The intermediate files were still huge. Writing them through a mount to the storage machine saved local scratch space, until the mount became unreliable.

The decoder itself became part of the work too. In my tape-decode fork, Codex investigated and wrote fixes for a bunch of issues where multithreaded decoding would jam up, with me overseeing the work. So this was not just asking an agent to run somebody else’s program. It was also digging into why the program got stuck and changing the code. That does not mean every threading problem is gone, but those fixes are another part of how this setup kept moving forward.

A decoder can still be running while its output is no longer reaching the destination. A mount process can exist without being useful. Successfully launching a command tells me surprisingly little about whether the job is going to finish.

So there are checks for actual file growth, child-process logs, transfer lengths, and expected picture-field counts. There is also a chunked path: close a piece locally, transfer it, verify it, reclaim the space, and assemble the pieces in order afterward.

This is where the long journal earns its keep. Each tape has records of the current files, commands, attempts, and results. The procedures explain which machine does what and what counts as completing a stage. A later Codex conversation can inspect that state and continue instead of needing me to remember every detail.

I also use the shit out of goals in Codex for this. I give it an outcome to work toward, like getting a tape ingested, preserved, decoded, and reviewed, with the records updated. The goal says what I want finished; the repo’s instructions, procedures, and journal give it the details needed to keep working toward that. There are a lot of intermediate steps in this setup, and I do not want “the decoder started” to be mistaken for the job being done.

The procedures change because the work changes. A failed mount leads to another check. A misleading preview changes the review instructions. Something that worked for a short test turns out to need a different approach for a whole tape.

I am pretty amazed Codex can deal with all of this. I can ask it to help capture, preserve, decode, review, export, and update the records across this weird setup. I still have to physically put the tape in the VCR. There are limits to the agent thing. 😅

And a process starting successfully is not successful preservation. Even finishing an encode does not establish that the source is safely retained, the audio is right, or the important sections were found. That is why the notes have become rather fussy about “done.”

Cheaper does not mean cheap

People have been asking what this approach costs. I went back through my purchase emails, and even the partial breakdown is a reminder that an ordinary VCR does not make the whole project cheap.

Item	Cost / replacement allowance
Sony SLV-688HF VCR, including shipping and tax	$69.13
Capture computer, salvaged from the junk pile at work	$0
Samsung 980 1 TB NVMe scratch SSD, already on hand	$0 additional spending; allow about $300 for a new 1 TB replacement today
Capture-PC boot SSD, PCIe NVMe adapter, BNC cable, tap lead, and SMA/BNC adapters, grouped in one shipment	$100.49
S-Video-to-BNC splitter shipment	$8.77
SMA DC-block shipment	$27.38
Three PCIe capture cards for CXADC, including one dud	$90.02
JLCPCB amplifier prototype batch: five PCBs, two assembled	About $140
M5Stack Atom Lite ESP32 controller with built-in IR	$14
3D-printed stand for aiming the IR controller at the VCR	Cents
VCR power meter for transport-state monitoring	About $14
Estimated additional spending for my setup	About $464
With a current replacement allowance for the scratch SSD	About $764

Apart from the scratch SSD’s explicitly marked replacement allowance, these are historical purchase amounts for my setup, including things bought while experimenting, not a recommended shopping list. The three matching AliExpress card orders show $30.87, $31.48, and $27.67. One card was a dud, so I ended up buying three; that is my spending, not a requirement to buy three cards. The emails also carry a shipping-fee exclusion notice, so those amounts may not capture every charge.

The amplifier batch is about $140 from memory, pending the final invoice. I am counting the whole batch rather than dividing it into a misleadingly cheap single-board price. The Amazon shipment totals also differ from the initial order confirmation. With the $14 IR controller, roughly $14 power meter, and cents for the printed stand, call this roughly $464, with some accounting still to reconcile.

The scratch SSD was another thing I already had, not another free component somebody else can assume they will find. Current prices are wild. On September 13, 2026, a Best Buy marketplace listing for a Samsung 990 EVO Plus 1 TB showed $299.74, sold and shipped by Merchant Expo LLC. I am using roughly $300 as a current replacement allowance, before tax. That is a newer model, not the Samsung 980 actually in my machine, and one listing rather than the cheapest-drive claim or a market average.

The capture computer came out of the junk pile at work. It cost me nothing. Maybe call it a $100 used computer if you had to find something similar, but that is my guess, not a checked price. Including both that computer estimate and the scratch SSD allowance puts this partial budget around $864. The separate boot SSD and PCIe adapter I bought are already counted in the shipment above; the 1 TB scratch drive is not bundled into that amount.

The other computers already on hand, bulk storage, tools, label printer and its tape, smiley-face stickers, electricity, and AI usage are outside this subtotal too. Starting from nothing would be a different budget.

For comparison, the TBC-equipped decks I found on eBay while writing this were roughly $700–$1,600, depending on model and claimed condition. A JVC HR-S9911U listing marked sold showed $900, a seller-described rebuilt HR-S7600U was listed at about $1,150, and a Panasonic AG-1980 listing describing capacitor replacement showed about $1,600. Those last two are advertised prices, not verified sale amounts, and this is a small sample rather than a market average.

My additional spending comes to around $464, helped by a free computer and a scratch SSD I already owned. Allowing for buying those two items brings this partial budget closer to $864, which overlaps the lower end of those TBC-deck examples. There is still a lot of assembly, troubleshooting, and existing equipment behind that number. The eBay prices are for the deck alone too, so neither side is a complete capture setup priced from scratch.

Still growing

This started as a way to avoid paying for a fancy VCR. It has become a small preservation workflow, with considerably more DIY than the price comparison alone suggests.

I would not present it as a polished setup for somebody else to install. Parts work well, parts need babysitting, and the next tape can still expose another bad assumption.

I am keeping my own pipeline repo and tape collection private, but you do not need my exact repo to try this. I think this post should give you a starting point for making your own: write down what your machines do, keep a journal, and build up scripts and checks as you go. Steal whichever tricks here are useful, adapt them to your setup, and add the ones you figure out yourself. That is pretty much how mine grew.

But the payoff is that I can now put a tape in, start a new Codex thread in the repo, and give it an ingest goal.

It reads the instructions and notes, checks that there is room for the capture, rewinds the tape through the IR controller, and watches the power meter to confirm the rewind has finished. It gets the capture running, sends Play, and records the whole tape, using the meter to detect end-of-tape. Then it can carry on with preserving the files on the NAS, decoding, reviewing, and writing up what it found. Capture everything first; sort out the contents afterward.

There is a lot behind that little request, and it can still need my help when something goes wrong. But I no longer have to manually coordinate every command on every machine just to get through a tape. The scripts, checks, and long journal are what let a fresh thread pick up the job.

That is the part I find amazing. This sat on my someday pile for about two years, and now I can toss in a tape and ask Codex to get started. I did not set out to build all of this. I wanted to see what was on the tapes.

Weftan, a backup plan that became a Rust rabbit hole

Mon, 07 Sep 2026 10:34:51 -0700

Weftan started as a fun backup plan. If TALA died hard, could I still have the layout engine around in some usable form?

TALA arranges diagrams for D2. Recovering its behavior and rebuilding it in Rust seemed like an interesting thing to try. That eventually became Weftan, and then a larger work-in-progress project called d2-rs.

Weftan was faster than the Go version of TALA when that was an external proprietary binary. The slightly annoying part is that TALA built into D2 was faster still. So naturally I started thinking about rewriting more of the surrounding thing too.

Recovering the behavior

The early work used TALA release artifacts: symbols, debugging information, disassembly, and comparisons with the executable. Reconstructed Go code informed the Rust implementation. This was recovery work, and the provenance matters; I did not independently invent the same layout engine by looking at a few pictures.

Then TALA’s source became available in D2 v0.9.0. That changed the situation quite a bit. There was now an upstream source implementation to follow and compare against, instead of relying entirely on recovered behavior.

There is a lot hidden inside “put the boxes somewhere sensible.” Containers, labels, edge routes, and the order in which equally reasonable choices get considered can all affect the final result. Two diagrams can look basically the same while disagreeing on a lot of coordinates.

So the useful comparison was to give both implementations the same measured graph and seed, then compare the actual geometry. When something differed, there was a smaller question to investigate than “why is this picture weird?”

Matching a corpus

The recorded Weftan baseline reached exact matches on all 120 successful single-seed comparisons in the checked-in corpus. That included 2,964 boxes and 974 routes, along with their corresponding labels and node icons. One large-grid case hit the work limit in both engines and was recorded separately.

I was happy with that. I also want to keep the size of the claim attached to it: those are the cases in that corpus, with that reference version and comparison setup. It does not mean every possible D2 input will behave identically.

Being able to reproduce a mismatch is useful even before it is fixed. It gives the next round of work an actual target. Otherwise it is very easy to keep improving the appearance of one example without knowing whether another example got worse.

The comparison machinery ended up being a useful result of the experiment in its own right.

The plugin tax

Weftan plugs into D2 as an external layout process. D2 has to pass a graph across that boundary and get the result back. Built-in TALA does not pay that same process and protocol cost.

There are two different comparisons here. Against the old external proprietary Go TALA binary, Weftan was faster in my testing. Against TALA built into D2, the end-to-end measurements went the other way. The baseline changed, and that distinction matters.

Plugin overhead is a plausible explanation for at least some of the gap with built-in TALA, but I have not isolated it well enough to say it accounts for the entire difference. “Rust” on the box does not make the complete workflow faster by itself.

That leaves Weftan in a perfectly reasonable place as a fun experiment. The backup-plan motivation got me started, and I got a recovered engine, comparisons, and a much closer look at the behavior out of it.

It also left me curious about removing that boundary altogether.

Following upstream in Rust

The next rabbit hole is d2-rs: a pure-Rust implementation targeting native execution and WebAssembly, with the layout engines running in the same process. The TALA work from Weftan has moved into that effort.

I am interested in following upstream behavior and exploring parallel execution where it makes sense. Having the surrounding pipeline in Rust should also make it easier to investigate costs without the external plugin boundary sitting in the middle of every layout.

This is still under construction. There are working source-processing and rendering paths, but plenty of compatibility work remains. A working SVG example is a nice milestone; it does not establish that the renderer, all the diagram features, or the other engines are interchangeable with D2.

I also do not have a speedup to announce for the rewrite. That is something to measure as it develops.

For now, Weftan is public. It was a fun backup plan, it taught me things, and it apparently gave me another project to work on.

Keeping comma hardware repair answers somewhere findable

Sun, 06 Sep 2026 10:34:51 -0700

One of the things I keep working on is the unofficial comma hardware documentation. Somebody has a broken device, people help investigate, and eventually there is an answer somewhere in Discord.

Then somebody else has the same problem.

Discord is an annoying place for the answer to live. Finding the right conversation is work, the useful detail might be several messages away from the original question, and a person searching the web may never find it at all.

So I keep trying to get those cases into a public document that search engines and AI assistants can read. Especially AI assistants, because people already ask them how to fix things. I want the actual repair knowledge to be available when somebody asks about their broken comma.

That is a big part of why I do it this way. An answer buried in Discord is not much help if the tool somebody is asking cannot get to it. A public document gives it something to find, read, and point back to. It does not guarantee a correct answer, but at least the specific cases and their limitations are there to work from.

The useful part is the follow-up

The document is organized around cases: what the person sees, what might be wrong, and what has helped other people with that particular problem. It has grown quite a bit. I do not expect anybody to sit down and read the whole thing in order.

Recent additions and updates include more fuse repair reports, a fastboot-only recovery case, and another cable gotcha. These are the small details that can save somebody a lot of time if they are findable at the right moment.

Fuse cases are a good example of why the reports matter. A suspected fault and a confirmed repair are different pieces of information. Somebody coming back with “that fixed it” makes the original discussion much more useful. An unsuccessful repair is worth recording too, especially if the symptoms seemed to fit.

The repair work belongs to the community members doing it and reporting their results. My contribution here is collecting the cases, keeping the references, and updating the guidance as more information comes in.

Similar-looking problems need different answers

The fastboot-only case covers a specific recovery situation on the comma three and comma 3X. A device can still show up in fastboot while being unable to enter the EDL mode used by the current flasher. Seeing a USB device on the computer does not establish that the flasher can talk to it in the mode it needs.

That distinction needs to stay attached to the recovery notes. Otherwise a workaround for a particular failure starts looking like a general recommendation for everybody.

The comma four cable case is another good one. The included OBD-C cable does its intended installation job, but lacks the USB data lines needed for computer flashing. That is a pretty frustrating detail to discover while you are already trying to recover something.

Neither case needs a second set of instructions here that will drift out of date. The maintained document has the details and references.

Constant little updates

This work is mostly incremental. Another symptom, another result, a clearer explanation of which device a case applies to. Sometimes the useful update is just making it harder to confuse two similar situations.

I like having a place where those answers can accumulate. Whether somebody finds it through a search engine or asks an AI to help, I want them to end up at the relevant case, with the references and pictures available to check. The answer should have somewhere to point back to.

The docs are here. If you try a remedy from them, please report back whether it worked. That follow-up is part of what makes the next version of the document better.

I might need these docs myself soon. My own comma 3X did not boot when I tried powering it over USB. I have not figured out why yet, so I may get to find out how useful my own documentation is from the other side.

Could the Crazy Taxi Arcade TAS use one fewer passenger?

Mon, 10 Aug 2026 10:34:51 -0700

I got curious about whether a Crazy Taxi tool-assisted speedrun could be slightly faster with one fewer passenger.

I do not really have TAS skills. This was a “could that work?” rabbit hole that led to looking at passenger tables, destination selection, and the game clock. I have not made a faster run.

The tips are where the real TAS money is. Chaining Crazy Throughs and drifts can pile up money while carrying a single passenger. SJ’s run aims for the Crazy License, reaches $20,000 with four passengers, and uses the long Police Station trip for a huge combo chain. The run ends when the clock reaches zero. So the idea is to earn enough money from tips with fewer fares, then have less clock left to burn off.

The specific idea starts after the first drop-off at Cable Car Stop TOP, the plaza at the top of the hill. In SJ’s four-passenger route, the next pickup there goes to West Side Beach, followed by a passenger going to Popcorn Mania, and then one going to the Police Station.

My idea was to leave the plaza and grab the side-street passenger going to FILA instead of taking the nearby passenger to the beach. FILA puts me near the next Police Station pickup, so I could go from that drop-off to the final passenger without doing the beach-to-Popcorn Mania fare in between. FILA appears as “Shoe Rack” in the Android data I used for the investigation.

That is why three passengers seemed promising: skip a whole fare and its extra pickup and drop-off, while keeping the long Police Station trip for earning tips. The side-street detour and the changed route could affect traffic and combo opportunities, so I would still need to show that three passengers can earn enough. Fewer stops only help if the money works out too.

The clock is part of the problem

Picking up a passenger adds time to the Arcade clock. A successful delivery can add more. So changing the passenger route also changes how much time the game hands back to you.

The symbol-rich Android release was useful for investigating this. Its code shows a distance-based time allowance at pickup, with a multiplier that changes as play goes on, plus a delivery-rank bonus at drop-off.

In the calculation in my notes, a proposed three-passenger route adds about 12.56 fewer seconds than the four-passenger reference route. That comparison assumes the early-game multiplier and the best delivery judgement for each passenger.

That is less added game time, which made the idea interesting enough to investigate further. It is not a measured 12.56-second improvement in a TAS. Driving, passenger animations, and when the clock actually advances still have to be accounted for.

Which destination do you get?

The Android investigation also showed that passenger destination selection follows a counter. It cycles through three destination slots, with each slot lasting 600 passenger-manager updates, roughly ten seconds at 60 updates per second.

The choice is latched when the passenger event activates. Waiting until boarding finishes to reason about it would put the timing at the wrong point.

That makes route planning more interesting. Reaching a passenger at a different time can select a different destination, which changes the trip and the clock allowance. Picking three convenient dots on a map is not enough.

To look at this spatially, I forked the noclip.website viewer and loaded the GameCube Arcade city map with the extracted pedestrian pickup locations overlaid on it. Credit to noclip.website for the viewer; my additions turned it into a research map for this passenger-routing rabbit hole. It is much easier to reason about a side-street pickup when I can actually see where it sits in the city.

The extracted passenger records match the Android reference records, which is useful evidence for understanding the data. It does not prove that every timing behavior carries over to the GameCube executable.

Maybe later

The next step would be to verify the relevant behavior in the GameCube version and try the route in an actual TAS workflow. There is still a gap between a promising calculation and inputs that produce a faster run.

For now this is a curiosity with some code and notes behind it. I need to sit down with it when things are a bit less crazy and I have time to learn the TAS side properly. I might revisit it then. One fewer passenger sounded like a small question. Apparently it came with homework.

Agent-assisted bilingual subtitles for group watches

Tue, 16 Jun 2026 15:35:00 -0700

I made some agent-assisted subtitle tools for group watches.

The short version is that I wanted one subtitle track with English and Chinese at the same time. Some people read one language faster, some read the other faster, and switching subtitle tracks is not a group activity.

No repo link for this one. The subtitle files are not mine to redistribute, and the interesting part is the workflow anyway.

The group watch problem

The simple version sounds extremely simple:

Take an English subtitle file.
Take a Chinese subtitle file.
Put both lines into one subtitle cue.
Watch the thing.

Unfortunately, subtitle files are not just text. They are text, cue numbers, timestamps, line breaks, release-specific offsets, missing cues, extra signage cues, and sometimes a drift that slowly changes across the whole movie.

The first version I tried was basically a mux. If the English and Chinese files had matching cue numbers, use the English timestamp and stack the text together.

That worked well enough for one case. It was also obviously fragile.

The annoying version

The more interesting case had two subtitle tracks that did not line up with one fixed offset.

If it were just “add 12 seconds to every Chinese cue,” this would not be worth writing about. But the offset changed over time. It started around one value and gradually moved toward another.

So the script got more specific:

parse both subtitle files
treat the English track as the timing source
search for the best Chinese offset in local windows
smooth those offsets
remap Chinese cues onto the English timeline
merge Chinese text into the nearest overlapping English cue
preserve Chinese-only cues as their own events
write an alignment report for things that looked suspicious

The report was the important part. I did not need the script to pretend everything was perfect. I needed it to tell me where to look.

The useful output was not just the merged .srt. It was also a short review list:

low-confidence merges
Chinese-only cues
English cues with no matching Chinese text
local offset anchors

That made the workflow tolerable. Generate, skim the report, spot-check the weird timestamps, adjust the heuristics if needed, and regenerate.

Where agents helped

This is exactly the kind of small, specific tool where agents are useful.

I did not need a general subtitle product. I did not need a GUI. I did not need a library that handles every movie ever made. I needed one script for one group watch, and then a slightly better script for a different subtitle mess.

An agent can help make that cheap enough to bother with.

The first pass can be dumb:

same cue number -> same timestamp -> Chinese line + English line

Then the next pass can be less dumb:

local timing windows -> estimated offset -> remapped cues -> overlap match -> review report

The agent also made it easier to keep the code disposable. I did not have to lovingly design a subtitle framework. I could just ask for the thing I needed, run it, look at the bad spots, and ask for a better report.

If you want to do the same thing, point Codex, Claude Code, or Google Antigravity at this post and your own subtitle files. The post is basically the spec: one timing source, one translated track, local offset search, merged output, and a review report. That is enough for an agent to build the throwaway script for your specific case.

Anyway, it worked for the group watch. We had a very good time, nobody was lost or left behind!

codex-status-json

Tue, 16 Jun 2026 15:03:00 -0700

I made a small Rust CLI that prints Codex status as JSON.

The short version is that I wanted Codex conversations to know my account, model, and quota state without opening the interactive TUI and reading /status like a person.

This is mostly useful with Codex goals. I am on the Pro 20x plan, which is the $200/month tier, so sometimes I really do have quota to burn. If I can read the remaining quota first, I can make a goal with an actual target instead of guessing how ambitious I should be.

The missing command

Codex has /status in the interactive CLI. It shows useful stuff: account, model, context, writable roots, and rate limits.

That is fine when I am sitting in the TUI. It is less fine when I want another Codex conversation to ask the same question.

There is an upstream issue asking for this exact kind of thing: a non-interactive JSON/headless replacement for /status. The issue is still open as I write this. There is also codex doctor --json, which is useful, but that is a diagnostic report. It is not the same as “tell me my account/model/quota status right now.”

So I made codex-status-json.

The first version was worse

The first version did what you would expect from a tool born out of impatience: it opened Codex in a PTY, sent /status, waited for the panel to settle, captured the terminal output, stripped ANSI escapes, and parsed the text.

It worked. It was also clearly a bad long-term place to live.

Terminal scraping is annoying because every little UI change can break you. The parser has to care about box drawing, timing, update notices, trust prompts, terminal width, and whatever else the TUI happens to render that day.

Still, it was useful. It gave me JSON like this:

{
 "schema_version": 1,
 "account": {
 "email": "user@example.com",
 "plan": "Pro",
 "raw": "user@example.com (Pro)"
 },
 "model": {
 "name": "gpt-5.5",
 "details": "reasoning high",
 "raw": "gpt-5.5 (reasoning high)"
 },
 "buckets": [
 {
 "name": "default",
 "limits": [
 {
 "kind": "5h",
 "remaining_percent": 99,
 "resets": "14:44",
 "reset_at_unix": 1780350290
 },
 {
 "kind": "weekly",
 "remaining_percent": 70,
 "resets": "08:23 on 7 Jun",
 "reset_at_unix": 1780845815
 }
 ]
 }
 ]
}

That was enough to unblock the thing I wanted.

The less cursed version

A few days later, I switched the default backend to Codex’s app-server.

That is a much better source than scraping the TUI. The tool starts:

codex app-server --listen stdio://

Then it sends JSON-RPC calls for:

account/read
account/rateLimits/read
config/read

That gives it the same kind of account, model, and quota information without pretending a terminal status panel is an API.

Important caveat: Codex app-server is documented, but it is also described as experimental and may change. So this is still not a supported codex status --json. It is just a more sensible workaround than screen scraping.

The PTY backend is still there as a fallback:

codex-status-json --backend pty

But the default is now:

codex-status-json --backend app-server

This is a bridge

I do not expect this to be the forever answer.

Codex’s usage and billing surface is still moving. OpenAI has already introduced things like banked rate-limit resets, so the status shape is not just “two counters and call it done.” There are plans, windows, reset times, secondary buckets, and whatever else comes next.

So I can see why OpenAI might not want to commit to a stable status JSON contract yet. That is fine. I still wanted something local that worked now.

This is the same kind of bridge as a lot of these small tools: use the available pieces, make the annoying gap tolerable, and be happy if the official thing eventually replaces it.

The repo

The project is here:

https://github.com/nelsonjchen/codex-status-command

Install it locally with:

cargo install --path .

The README has the options and limitations. The tests include a captured /status fixture so the text parser can at least fail loudly when Codex changes its display.

I would still rather have an official codex status --json. Until then, this is small, local, and useful enough.

Moving Gmail Send mail as to Cloudflare SMTP

Tue, 16 Jun 2026 14:13:00 -0700

I just finished moving my Gmail “Send mail as” setup completely to Cloudflare Email Service’s native SMTP submission. I can send out emails under my mindflakes.com domain.

The short version is that I had built a small Rust SMTP-to-Cloudflare REST API relay in April. Today I noticed Cloudflare had shipped native SMTP submission.

Part of the appeal here is minimizing vendors. Cloudflare already hosts the domain, handles incoming mail through Email Routing, and now can handle outgoing mail through Email Service too. For a personal domain, that can be cheap or free, and it is a nice alternative to paying for Google Workspace when a personal Gmail account is otherwise fine.

The April hack

In April, I saw Cloudflare’s public beta announcement for Email Service and got excited about using Cloudflare for both inbound and outbound personal-domain mail.

Then I got sad. I wanted Gmail to send mail for one of my domains through Cloudflare Email Service, but I did not see a native SMTP submission path I could just point Gmail at.

Gmail’s Send mail as feature wants a normal authenticated SMTP server. You give it a hostname, a port, a username, and a password. Gmail talks SMTP. It does not want to be handed a REST API endpoint.

At the time, the useful Cloudflare sending shape I had in front of me was the raw MIME REST API. Cloudflare had teased SMTP in the earlier private-beta era, but the public thing I could actually use was not SMTP.

So I made smtp2cf.

It was intentionally narrow:

accept authenticated SMTP from Gmail
write the raw MIME message to disk before returning 250 OK
forward that raw MIME to Cloudflare’s send_raw API
retry transient failures
move permanent failures to a dead-letter directory
stay small enough to run on a cheap little VPS with an IPv4 address, because that is what I already had

Mostly, I wanted to avoid operating a real mail server for a personal domain.

The temporary bridge

This is also one of those places where Codex changed what felt reasonable to do.

Before, I probably would have waited for Cloudflare’s rollout or grudgingly set up a heavier mail stack. But as far as I could tell, there was no useful public date for SMTP support. So I was impatient and made the bridge I wanted.

My first smtp2cf commit was April 18, 2026. I had the repo cleaned up and documented by April 20. It was not a forever project. It was a practical way to bridge the gap until Cloudflare shipped the official SMTP path.

Cloudflare’s docs now have a changelog entry dated June 8, 2026: Authenticated SMTP submission now available in beta. That means the relay did its job and can leave the live path.

The new setup

Cloudflare’s SMTP docs and Gmail’s Send mail as docs line up pretty directly:

host: smtp.mx.cloudflare.net
port: 465
security: implicit TLS
username: api_token
password: a Cloudflare API token with Email Sending access

I generated a token, changed the Gmail Send mail as SMTP settings, sent a test message, and watched it go through Cloudflare without my relay in the middle.

That is the best kind of infrastructure migration: delete a moving part and then immediately wonder why you ever had to own it.

The now historical Rust thing

The repo is still here:

https://github.com/nelsonjchen/smtp-2-cf-email-service

It is now mostly historical for me.

I still like a few parts of it. The durable spool is the right instinct. Returning 250 OK before the message is safely written somewhere is how you earn a future debugging session. The Gmail header cleanup is gross but also a useful reminder that “raw MIME” does not always mean “whatever one mail system happened to emit is acceptable to another one.”

My past email setups

This domain has had a few outgoing mail setups over the years.

Way back, I used DreamHost for Gmail’s Send mail as SMTP server. That mostly worked, but it was still another provider in the path.

Later, I used Zoho Mail as a sort of middleman mailbox. Gmail would send as the domain through Zoho, and Zoho would receive mail and forward it onward. I was not happy with either side of that. Incoming forwarding would sometimes fail to take, delivery rates sucked on Zoho, sending email out through Zoho SMTP was a shitshow, and traffic from Zoho to Gmail was a shitshow too.

The most annoying symptom was the delivery report email. I would get an email with a ZIP file of delivery-rate data, and the vibe was never “everything is fine.” It was more like “congratulations, your mail technically attempted to exist.”

Then came smtp2cf, the Rust relay in this post. That got me away from Zoho and let Gmail hand mail to my own tiny SMTP bridge, which then sent through Cloudflare’s raw MIME API. It was better because it was mine and inspectable, but it was still a moving part I had to run.

This Cloudflare SMTP setup is the first version that feels like the shape I wanted all along: Gmail UI, personal domain, Cloudflare for the domain and mail plumbing, and no mailbox vendor sitting in the middle.

Anyway, that is the trip report. I am happy this one is boring now.

Plainize Clip

Sun, 14 Jun 2026 23:05:00 -0700

I made Plainize Clip, a tiny macOS app that cleans the current clipboard and quits.

The short version is that I still wanted the old Plain Clip shape: copy weird text, launch the app, paste something boring.

Not a clipboard manager. Not a menu bar app. Not a background process sitting around in RAM forever. I launch it through Spotlight, it rewrites the pasteboard as plain text, and then it exits.

The annoying clipboard problem

Copying text on macOS is often not just copying text.

Sometimes it brings formatting. Sometimes it brings weird invisible characters. Sometimes it brings smart quotes, hard-wrapped lines, non-breaking spaces, tabs, and whatever else got dragged along from a PDF, web page, chat app, or ticketing system.

Usually I do not want a whole clipboard workflow. I just want the pasteboard to stop being annoying.

So Plainize Clip does the small version:

Read the general pasteboard.
If there is text, clean it.
Write back plain text only.
Quit.

That is the whole normal launch path.

Launch, clean, quit

The important behavior is that Plainize Clip is faceless.

There is no Dock icon to manage after the cleanup. There is no persistent process. There is no history database. If the current pasteboard has text, it cleans it with the saved preferences and replaces the pasteboard contents with text.

The prod was Apple’s Rosetta clock. Apple says Rosetta will remain available as a general-purpose Intel app translation tool through macOS 27, and then becomes limited starting with macOS 28. So an Intel-only utility like the old Plain Clip had a stronger “you should probably stop depending on this forever” vibe.

Choosing Swift/AppKit/SwiftUI was not a deep architectural statement. I just wanted to follow Apple’s current recommendations for a small Mac app.

The GitHub releases are unsigned, so macOS does the usual “unknown developer” dance. Apple’s support page documents the Open Anyway flow. This is annoying, but paying Apple for a Developer ID certificate for a small personal clipboard cleaner is also annoying.

Reverse-engineering because Codex made it cheap

I also ended up reverse-engineering the old app more than I normally would have.

Codex let me reverse-engineer stuff I otherwise would not have lifted a finger for. The local plain-clip-re workspace became the notebook for it: static inventory, Ghidra headless exports, behavior notes, and then a recovered map of what the old app actually did.

The useful bits were not mysterious. They were just tedious:

normal launch cleans the pasteboard and quits
holding Shift at launch opens preferences
the command-line wrapper has switches for the same cleanup options
the cleaner reads string content from the general pasteboard
after cleaning, it replaces the pasteboard with string-only content
the pipeline handles line endings, whitespace, hard wraps, blank lines, smart quotes, tabs, Unicode normalization, and ASCII conversion

That was enough to rebuild the behavior intentionally instead of just making “a clipboard cleaner, probably.”

Plainize Clip is still a separate implementation. I did not reuse the old app’s code. The reverse-engineering work was mostly there to answer boring compatibility questions without relying on memory.

The cleanup switches

Plainize Clip has preferences because text cleanup is personal enough to need switches.

The defaults do the stuff I usually want: trim whitespace, remove empty-line noise, join hard-wrapped prose, replace tabs with spaces, collapse repeated spaces, remove invisible control characters, and turn smart quotes and typographic dashes into simpler punctuation.

There are also options for Unicode normalization and ASCII conversion. The ASCII path tries to romanize non-Latin scripts before filtering, so it does not turn a clipboard full of CJK, Korean, Arabic, Hebrew, or Cyrillic text into an empty string. It is best-effort, not magic.

Preferences open by holding Shift while launching, or from Terminal:

open -a "/Applications/Plainize Clip.app" --args --preferences

Save the settings and it runs one cleanup pass immediately. Cancel does not.

The Plain Clip context

Plainize Clip borrows the spirit of Plain Clip by Carsten Blüm: stay invisible, do one job, quit.

It is independent from Plain Clip and is not affiliated with Blüm.

The context is that I liked Plain Clip, but the old archived 2.5.2 build I had found was Intel-only, the source release situation was unclear, and Blüm’s current site no longer listed those legacy Mac apps by default. His site says older macOS apps moved out of the normal public listing, and that interested users can contact him for specific versions.

Blüm later clarified to me that Plain Clip is still in daily use by him and that he has resumed work on it, including an Apple Silicon build. As far as I know, that newer build is still an on-request thing rather than something posted publicly.

My approach was different: Plainize Clip is open on GitHub from the start. It remains my separate modern build of the shape I wanted.

The repo

The project is here:

https://github.com/nelsonjchen/plainize-clip

The releases are here:

https://github.com/nelsonjchen/plainize-clip/releases

The README has the download notes, unsigned-app warning, preferences, CLI wrapper, build command, and tests.

Blocking spammy caller names with VoIP.ms Call Hunting

Wed, 20 May 2026 19:30:00 -0700

I made a small Rust VoIP.ms CNAME blocker.

The short version is that I wanted to block a caller by name instead of by phone number.

This sounds like it should be an option in VoIP.ms. For all the features VoIP.ms has, this is not one of them. So I made one for cheap.

The missing filter

VoIP.ms has a pretty capable CallerID Filtering feature. You can filter on specific caller ID numbers, phone book groups, anonymous callers, calls that do not match the North American number format, STIR/SHAKEN attestation level, and wildcard patterns like area-code-ish blocks.

That is all useful, but these asshole spam callers and scammers do not politely stick to one number. They rotate numbers. Blocking one just means the next call comes from another one. What I wanted to block on was the caller ID name.

VoIP.ms has CNAM support too, but as far as I can tell it is for displaying names, not filtering on them. Their blog post on stopping spam calls talks about filtering numbers, area codes, and anonymous callers, while describing CNAM as a way to identify callers before picking up. The Caller ID wiki page describes incoming Caller ID name lookup as an optional per-DID setting that can display a caller name for US and Canadian callers.

There is also a VoIP.ms Community Forum thread where someone asked for exactly this: blocking repeated callers by CNAME/CNAM because the caller used a pile of different numbers. A VoIP.ms staff reply said the feature had been suggested before and that there was no ETA.

So, yes, this is apparently not just me being weird. At least not uniquely weird.

The Call Hunting trick

VoIP.ms has another feature called Call Hunting. A DID can try multiple destinations in order.

So I made a tool that registers to VoIP.ms as a SIP endpoint and sits first in a Call Hunting group. It checks the incoming caller name and then does one of two things:

If the caller does not match the block list, it rejects the call with 486 Busy Here.
If the caller matches the block list, it answers, plays SIT tones plus a disconnected-style message, and hangs up.

The 486 Busy Here part is the whole trick. For normal calls, VoIP.ms treats the first destination as busy and keeps going to the next Call Hunting member, which is the real phone path. The blocker gets out of the way.

For blocked calls, the blocker answers the call. Once it answers, the hunt stops. My actual phone never rings.

Why Twilio Lookup got involved

The first version matched the caller name that VoIP.ms sent in the SIP call.

Then the field I wanted was not the field I got.

For the PCH-style caller I cared about, the VoIP.ms-provided name showed up as a generic city/state-style value. That is not useful enough to block on. Meanwhile, T-Mobile Caller ID showed PCH for the same number.

Searching for PCH phone number scam gets you Publishers Clearing House, plus warnings about PCH impersonator scams. PCH has its own scam-report page saying scammers misuse the PCH name to pretend people won prizes and demand money or personal information. The FTC also has an alert specifically about PCH impersonators.

I do not know where T-Mobile got that name, and I am not claiming Twilio uses the same source. My understanding is that CNAM is not one magic database anyway. Different carriers and lookup providers can have different data, different caches, and different levels of freshness. Twilio’s database may simply be more thorough for this case. VoIP.ms was not giving me the useful label, so I added optional Twilio Lookup.

If configured, the blocker can ask Twilio Lookup for the caller name and match against that instead. If Twilio does not return a useful name or the lookup fails, the call is allowed through.

The matching itself is intentionally boring:

BLOCK_CNAME_PATTERNS=pch

Those patterns are case-insensitive and match on token boundaries, so pch can match PCH without accidentally matching some longer unrelated word. There is also a regex option for stranger cases.

The repo

The project is here:

https://github.com/nelsonjchen/cname_blocker_voip

It is not a PBX. It registers outbound to VoIP.ms, checks caller name information, optionally asks Twilio Lookup, returns 486 Busy Here for normal calls, and only answers calls it wants to block.

I expect this exact piece of software to be useful mostly for myself, but I do hope people use the code as a starting point. After I posted it in the VoIP.ms thread, someone asked about using it for a simple “press 1 to continue” style flow instead of dropping the call. I do not need that, and the code is not designed for it, but it is not a ridiculous direction. With an agent like Codex pointed at the code and tests, that kind of custom call flow seems very reachable. In a very small and specific way, this program is already acting like a focused PBX.

It is written in Rust, and I deployed it to my TrueNAS box. Resident memory is around 5 MB.

Codex 5.4 Medium helped write a lot of it.

The README has the actual setup steps.

The tests do not need real VoIP.ms credentials either. There is a local fake registrar/integration test path that sends matching and non-matching calls, checks that blocked calls get answered, checks that normal calls get 486 Busy Here, and verifies RTP audio makes it through.

Anyway, it works for the annoying thing I wanted to stop.

I still think VoIP.ms should offer caller-name filtering directly. Until then, this does the job.

Unofficial MakerWorld PMM OpenSCAD Reference

Mon, 04 May 2026 08:15:00 -0700

I made an unofficial MakerWorld PMM OpenSCAD reference.

The short version is that I wanted Codex to help me make MakerWorld-customizable OpenSCAD models, and I did not want it guessing from old forum posts, UI behavior, and whatever I happened to remember at 1 AM.

This started because I wanted to use agents on my own OpenSCAD projects without re-teaching the same MakerWorld-specific weirdness every time.

The documentation problem

MakerWorld’s Parametric Model Maker is genuinely useful. OpenSCAD plus a web customizer is a great fit for 3D-printable stuff where the interesting part is “same idea, slightly different dimensions.”

If you have not run into it before, OpenSCAD is CAD by writing code. You write variables, modules, and geometry operations, and it turns that into a model. That makes it very good for customizable prints, and very annoying when a target platform has extra rules.

Unfortunately, the OpenSCAD side of PMM is documented in a very internet way. Some of it is in release posts. Some of it is in support replies. Some of it is in the actual web app. Some of it is people trying things and reporting what happened.

There is a Bambu forum thread literally titled Any Documentation on Parametric Model Maker?. The first post asks where to find the requirements for making customizable models. That felt familiar.

For a human, scattered docs are annoying. For a coding agent, they are a great way to get plausible garbage. It might write OpenSCAD that looks fine locally but misses MakerWorld’s actual rules. It might invent a PMM feature because it saw something sort of similar somewhere else.

I do not want an agent inventing // preview[...] as a PMM feature. I want it to know that // color is a thing, // font is a thing, mw_plate_N() is a thing, and default.svg is not just some random example filename.

So I made a reference.

What I collected

The repo is basically a practical map of the PMM OpenSCAD surface:

the PMM OpenSCAD API
an agent workflow for converting normal OpenSCAD into PMM-ready OpenSCAD
gotchas that agents are likely to mess up
compatibility rules
public source snapshots
patterns and checklists
a generated docs site

The small examples are the important ones:

accent = "#FF0000"; // color
font_name = "Roboto"; // font

module mw_plate_1() {
 // printable plate here
}

module mw_assembly_view() {
 // preview-only assembly here
}

svg_file = "default.svg";

None of that is hard once you know it. The problem is making sure the agent knows it before it starts “helping.”

I also made a PMM font index. It has 1881 MakerWorld PMM font families and 8267 exact PMM font strings. This got more elaborate than I expected, but fonts are exactly the kind of thing where an agent can confidently choose something that works on your machine and then does not work where the model actually runs.

Why agent-first

This is not meant to replace the official PMM UI or be a polished tutorial for someone’s first customizable model. It is mostly a pile of receipts and instructions for agents.

Point Codex or another coding agent at the repo, then ask it to adapt a .scad file for MakerWorld. The repo gives it rules to retrieve before it starts editing:

flatten local includes when PMM probably will not have them
do not remove bundled PMM libraries like BOSL2 just because arbitrary local includes are risky
use PMM upload defaults like default.svg and default.stl
add // color and // font only where those controls are intended
treat multi-plate output as a publishing decision, not just a module name
cite whether a claim came from an official app endpoint, official release, employee reply, community report, or inference

That last part matters. I do not want a pile of agent-generated confidence. I want the agent to say why it thinks something is true.

The repo

The project is here:

https://github.com/nelsonjchen/unofficial-makerworld-parametric-model-maker-openscad-docs

The generated docs site is here:

https://nelsonjchen.github.io/unofficial-makerworld-parametric-model-maker-openscad-docs/

This is unofficial, not endorsed by Bambu Lab or MakerWorld, and not a license grant for any font, library, model, or web asset. It is practical documentation from public sources and reverse-engineering.

It is also probably incomplete or wrong in places. PMM keeps changing, and some of this stuff only becomes obvious after someone trips over it.

Anyway, this mostly exists so my agents stop making the same MakerWorld OpenSCAD mistakes. If it saves someone else from digging through forum threads while trying to publish a customizable model, great.

Cisco Catalyst 9800-CL on GCP: what a fight

Fri, 20 Mar 2026 13:15:00 -0800

I finally wrote up my notes on getting a modern Cisco Catalyst 9800-CL running on Google Cloud Platform.

The short version is that this was much more of a fight than I expected.

Cisco’s public GCP-facing material still nudges people toward Google Cloud Marketplace, but the Marketplace offerings I found were outrageously old: 16.12.1 and 16.12.2s from the 2019-2020 era, plus 17.2.1 and 17.3.5a from the 2020-2021 era.

That is a pretty bad situation if what you actually want is a modern WLC on GCP. The version I actually worked from was 17.15.04d, which Cisco’s software portal lists with a release date of 19-Dec-2025. Cisco’s public guidance still points at a path that appears stuck on software families from years ago, and it was not a good starting point for getting a current controller running.

The current downloadable image path is possible, but it turned out to involve more image surgery and boot-path archaeology than I had hoped for.

I put the actual handoff-style guide here:

Cisco Catalyst 9800-CL on GCP guide

That guide covers:

where to get the qcow2
why Marketplace is not a great baseline right now
what actually worked to get the controller reachable
how I validated it was really up
how to adopt an AP into the supported public-cloud path

One of the main conclusions is that Cisco’s built-in GCP bootstrap path seems to create real first-boot state that the appliance expects, which helps explain why simply editing /varied/iosxe_config.txt offline was not enough on a pristine raw image.

Anyway: what a fight. But at least the notes are in one place now.

Bambu Lab Store Filament Tracker

Wed, 11 Feb 2026 20:10:00 -0800

I’ve been running a Bambu Lab Store filament tracker at bbltracker.com.

I recently got into 3D printing. A lot of what I print is practical: last-millimeter adapters, things like robot vacuum ramps around the house, and other tiny fixes that make daily life less annoying.

Bambu Lab released the P2S, their flagship midrange printer, and I picked one up after hearing how polished their ecosystem is. People call them the Apple of 3D printers, and I can see why. The hardware is solid, but the software and ecosystem are really what make it shine.

There is also a bit of a format war around filament. Bambu sells filament with RFID tags that are not easy for third parties to mimic. The RFID flow makes loading filament easy and automatically applies Bambu-tuned profiles. Some people feel those profiles run too fast, but for my functional prints, they have worked great.

Where the frustration started

I like having color options, and color can be functional too. But stock has been rough for a while, even after the holiday season.

That rough stock situation (and the need for a tracker) is really a symptom of how popular Bambu has gotten, especially over the holidays:

Source: Reddit comment

Then there is the constant bulk-sale mechanic on the Bambu Lab Store: buy 4+, get a strong discount, buy more, save more. In practice, that falls apart when the combinations you want are never all in stock at the same time.

At the time, the store UI also made this harder than it needed to be. Out-of-stock options were not clearly crossed out, so building a cart felt like guesswork.

That made me angry, but in a constructive way. So I built a tracker.

What the tracker does

At a high level, the tracker monitors store inventory over time, keeps historical snapshots, and renders a dashboard so I can quickly answer:

What is actually in stock right now?
What appears stable versus constantly draining?
What was the last major restock spike?
Are there ETA hints for restocks?
Is this likely to go out of stock soon?

I started with only the US store, then expanded to EU, CA, UK, AU, and Asia coverage.

I also reused a stock-quantity verification trick I had learned while documenting openpilot’s Toyota security key journey.

Building it fast (and keeping it moving)

I used a lot of Google Antigravity while building this. Web scraping is tedious work, and large language models helped a lot with iteration speed and the constant tweaks.

The codebase is not pretty in every corner, but LLMs have made it very workable to maintain and improve continuously.

Later, I added depletion-rate estimation so the tracker can make a rough guess about when something might go out of stock.

Things that happened after launch

After I released it and posted it on /r/BambuLab, a bunch of interesting things happened:

Bambu nerfed the original stock-count hole the tracker used.
For a while, counts were capped at 200 in the US view, which reduced visibility.
Later, that cap moved to 400 in the US store. Maybe my tracker’s utility was actually a factor in that decision? I do see traffic from China, where there is no Bambu Lab Store of the same codebase as far as I can see.
Bambu improved their UI and added clearer out-of-stock cross-outs.
I added more quality-of-life indicators, including “stable stock,” ETA context, and restock-spike visibility.
I added a GoFundMe and a nice patron funded a custom domain and hosting for the tracker.

Backend at a glance

Keeping this online reliably has mostly been a deployment and operations problem:

Google Cloud Run is heavily used for scheduled jobs, scaling, and reliability.
Cloudflare Pages hosts the output and leaves room for future expansion.

If you print a lot and buy filament in batches, this has been much better than manually refreshing product pages and hoping for the best.

Surviving a power outage with fiber ONT and PoE setup (Frontier FOX222)

Sun, 06 Jul 2025 18:12:00 -0800

For some reason, it’s been a bit hard to find information on how to setup my particular fiber ONT so that it can survive a power outage with a PoE setup.

While of course, one could power a whole house with solar and batteries, I wanted to find a way to keep my fiber ONT and router powered during a power outage without having to invest in a full home battery system.

Costco recently had on sale a small EcoFlow RIVER 3 Plus Portable Power Station, which I installed into my home network setup with the power station being next to my router.

It’s been wonderful as an UPS!

My setup is a bit peculiar as my Frontier FOX222 ONT is a bit far from the power station, but the power station is next to the router and there is an Ethernet cable running from the ONT to the router. The RIVER 3 is also next to the router.

The only way to provide power to the ONT without also having to plumb AC backed by the power station: A PoE injector on the Ethernet cable between the ONT and the router and a splitter at the ONT end to convert the PoE to 16V DC power for the ONT.

The FOX222 ONT is powered by a 16V DC power supply. There is an “Auxiliary Power Supply” port on the side of the power supply. Unfortunately, there is no printed information on what acceptable inputs are acceptable for this port. Nor is there documentation on the power supply itself on acceptable voltages. And size, that took forever and was inconsistent.

Information is also a bit scattered and/or some people have done permanent modications. Some sources have even said 12V is all you need but this is wrong! The information may be applicable to a different ONT model.

Through some trial and error and quite some Amazon returns, I’ve have an Amazon checklist of the requirements for what is needed.

My environment:

Frontier FOX222 Fiber ONT (Optical Network Terminal)
Cyberpower CA25U16V2 Power Supply

Requirements:

You need a PoE injector that can provide at least 30W. The power supply itself at the end of the chain is rated for 16V at 1.6A but it is unlikely to use the full 1.6A.
- TRENDnet Gigabit Power Over Ethernet Plus Injector, PoE+ (30W)
You need a PoE splitter that can provide 16V DC output or something close to it. The FOX222 ONT requires 16V DC input. It will not run with 12V DC input. The 18V DC output is within range and works.
- LINOVISION 30W Gigabit PoE Splitter to DC 5/9/12/18V Output, Wide Voltage Input
The power supply connector has a DIN-like connector on one end. It must be modified so that when there is no AC power, the ONT will still power on and provide data.
- The modification is easily reversible with a small piece of alumnium foil.
- Image from https://www.instructables.com/I-211M-L-ONT-Enable-Data-While-on-Battery-Power/
- If not modified, apparently the ONT will refuse to power on the data port. I did not care to try.
The “Auxiliary plug to the the power supply is a 4.0mm x 1.7mm barrel connector. I don’t know why some some sources say it’s a 4.5mm x 3.0mm barrel connector. Or have other bizzare sizes such as “1.3mm x 3.5mm”.
- 4.0mm x 1.7mm barrel connector
You’re adding some Ethernet cable segments. Some small thin cables is fine.
- Monoprice’s SlimRun cables seem to be excellent. I have a few 1 ft patch cables

Installation:

Plug the PoE injector into the wall and connect it to the router on the WAN side. Use the short Ethernet cables to add small segments as needed.
Connect the PoE splitter to the ONT on the other end of the Ethernet cable. Use the short Ethernet cables to add small segments as needed.
The LINOVISION PoE splitter has a 5.5mm x 2.1mm barrel connector male as an output. Connect it to the PoE adapter’s cable with the 4.0mm x 1.7mm barrel connector.
Plug the 4.0mm x 1.7mm barrel connector into the “Auxiliary Power Supply” port on the Cyberpower CA25U16V2 power supply.
Take out the wire between the ONT and the power supply and modify the plug as shown in the image above. This is done by stuffing in a small piece of aluminum foil into the barrel connector. It apparently shorts some pins together to allow the ONT to power on and provide data when there is no AC power.
Unplug the AC cord from the Cyberpower CA25U16V2 power supply.
Plug in the DC barrel connector into the power supply’s “Auxiliary Power Supply” port.
It should power on!
Do not plug in the AC cord! as if you plug it in, the ONT will reboot. This is actually bad. If the AC power goes out, the ONT will reboot. Just leave it powered by the DC power supply exclusively and from the PoE injector.

Setup Image:

Operation:

Leave the AC cord from the power supply unplugged. If you plug it in, power outages will disrupt service. You’re welcome to test this yourself once you have the setup working completely on DC power.

Things I don’t know:

Does this work the with the FRX523? It’s apparently some sort of successor to the FOX222.
Will there by long term issues with the ONT being powered by a 18V DC power supply?

Comments:

Questions and suggestions are welcome to https://bsky.app/profile/mindflakes.com .

References:

https://www.dmdtech.org/2024/11/30/power-your-frontier-and-others-ont-w-poe/
- Gave a hint that 16V is an absolute requirement
- Provided a hint to the LINOVision PoE splitter’s 18V output being acceptable
https://www.instructables.com/I-211M-L-ONT-Enable-Data-While-on-Battery-Power/
- Provided the image above
- Implies that the power supply is happy to take a 12V DC input but this is wrong for the FOX222 ONT, at least.
- Plug size is wrong, it’s 4.0mm x 1.7mm!
- Only thing right seems to be the modification of the power supply plug with aluminum foil

Talky Pet Watcher, a cat watching AI

Sat, 01 Feb 2025 06:08:55 +0000

Github Project: talky_pet_watcher

I’m allergic to cats 😢. But a friend asked me to watch their cat. My family was more than willing to help but to be safe, I wanted to also monitor the cat to make sure she didn’t get into any shenanigans that were too much.

With some cheap TP-Link Tapo cameras, I set up a system to watch the cat remotely and ensure she was safe while I could not be next to her. It’s a Telegram bot that also sends me a message whenever the cat is doing something interesting.

It watches multiple cameras and tries to aggregate the data to provide insights on the cat’s behavior.

Here’s the project description:

Talky Pet Watcher is a tool to watch a series of ONVIF webcams and their streams. It captures video clips, uploads them to Google AI for processing, and uses a generative model to create captions and identify relevant clips. The results are then delivered to a Telegram channel. This project is designed to help pet owners keep an eye on their pets and share interesting moments with friends.

This was slapped together in a few days, as I only watched the cat for two weeks.

I was also interested in Google Gemini and how cheap they claimed to be for analyzing video. I figured that this would be a good way to test it out.

What went well:

The system was able to capture interesting moments effectively.
It was able to concantate multiple camera’s POV to provide interesting views.
The cameras were cheap!
Putting the results on Telegram was a good way to share the results with the cat’s owner and friends.

What did not go well (and there are many!):

I did not implement history, so the system basically reported the same thing over and over again.
Getting reliable clips from multiple cameras when motion was detected was iffy due to the TP-Link Tapo camera’s iffy web servers. It would stall and halt a lot.
The Google Gemini AI could only analyze video clips at 1FPS. For an agile kitten, this can sometimes make wrong assumptions about what the cat is doing.
Implementation wise, bun was very crashy and I had to restart it a lot.
The connection to the camera also had to be restarted a lot.

If I had to do it again:

Use something that can pull from a NVR by relative timestamps. There was some Rust NVR but it was too complicated for me to set up in the small amount of time I had.
Implement a history system and maybe some memory bank system.
And so so much more! 😅

comma three faux touch keyboard

Fri, 13 Dec 2024 22:02:00 -0800

🦾 comma three Faux-Touch keyboard

Long arms for those of us with short arms from birth or those who can’t afford arm extension surgery!

Built a faux-touch keyboard for the comma three using a CH552G microcontroller. The keyboard is off the shelf and reprogrammed to emulat a touchscreen.

It’s a bit of a hack but it certainly beats gorilla arm syndrome while driving.

For $5 of hardware, it’s very hard to beat.

The journey started in January 2024 and ended in May 2024. Though it, I had to struggle with:

CH552G programming
Linux USB HID Touchscreen protocol
Instructions and documentation

It’s also been through that period of unintended usage. The Frogpilot project has had users who were surprisingly interested in using it as an alternative for some GM vehicles where there is no ACC button. They use the keyboard to touch the screen where ACC would be.

Part of the work also meant upstreaming to the the CH552G community. While I doubt it’ll have any users, it’s nice to give back to the community. There is now a touchscreen library for the CH552G.

The project is open source and can be found at:

https://github.com/nelsonjchen/c3-faux-touch-keyboard

There’s a nice readme that explains how to build and flash the firmware.

comma three faux touch keyboard

Fri, 13 Dec 2024 00:00:00 +0000

🦾 comma three Faux-Touch keyboard

Long arms for those of us with short arms from birth or those who can’t afford arm extension surgery!

Built a faux-touch keyboard for the comma three using a CH552G microcontroller. The keyboard is off the shelf and reprogrammed to emulat a touchscreen.

It’s a bit of a hack but it certainly beats gorilla arm syndrome while driving.

For $5 of hardware, it’s very hard to beat.

The journey started in January 2024 and ended in May 2024. Though it, I had to struggle with:

CH552G programming
Linux USB HID Touchscreen protocol
Instructions and documentation

The project is open source and can be found at:

https://github.com/nelsonjchen/c3-faux-touch-keyboard

There’s a nice readme that explains how to build and flash the firmware.

Replicate.com openpilot Replay Clipper

Thu, 12 Oct 2023 22:28:00 -0800

The replay clipper has been ported to Replicate.com!

https://replicate.com/nelsonjchen/op-replay-clipper

Along with it comes a slew of upgrades and updates:

GPU accelerated decoding, rendering, and encoding. NVIDIA GPUs are provided to the Replicate environment and greatly speed up the clipper.
Rapid fast downloading of clips. Instead of relying on replay to handle downloads sequentially in a non-parallel manner, we use the parfive library to download underlying data in parallel.
Comma Connect URL Input. No need to mentally calculate the starting time and length. Just copy and paste the URL from Comma Connect.
Video/UI-less mode. Don’t want UI? You can have it.
360 mode. Render 360 clips
Forward Upon Wide. Render clips with the forward clip upon the wide clip
Richer error messages to help pinpoint issues.
No more having to manage GitHub Codespaces. Replicate handles all the setup and cleanup for you.

Unfortunately, there is a cost. It’s a very small cost but technically Replicate.com is not free. Expect to drop a cent per render. Thankfully, you have a lot a trial credits to burn through and the clipper can run on a free-ish tier.

Colorized-Project: A Project-Color restoration.

Wed, 01 Feb 2023 22:28:00 -0800

GitHub Link: https://github.com/nelsonjchen/Colorized-Project

On IntelliJ IDEs, I’ve been using Project-Color to set a color for each project. This is useful for me because I have a lot of projects open at once, and it’s nice to be able to quickly identify which project is which.

Unfortunately, Project-Color hasn’t been updated in a while, and it doesn’t work with the latest versions of IntelliJ.

After a few months of using IntelliJ without Project-Color, I decided to try to fix it. I’ve released the result as Colorized-Project.

Thre are a few rough edges, but it’s mostly functional. I’ve been using it for a few weeks now, and it’s been working well. I’ve also submitted it to the JetBrains plugin repository as well.

It still needs some polish like a nicer icon, and I’d like to add a few more features.

Colorized-Project

Wed, 01 Feb 2023 00:00:00 +0000

GitHub Link: https://github.com/nelsonjchen/Colorized-Project

Remember your projects by color! Colorize the window somehow to differentiate your open projects. Updated and maintained fork of Project-Color.

Working on multiple projects simultaneously with Jetbrains products based on IntelliJ can be difficult, since they all look the same at a glance. This plugin solves that by giving the user the option to colorize each project’s window in a noticeable way.

GitHub Wiki SEE: Search Engine Enablement: Year One

Thu, 13 Oct 2022 21:04:00 -0800

It’s been near a year since I supercharged GitHub Wiki SEE with dynamically generating sitemaps.

Since then a few things have changed:

GitHub has started to permit a select set of wikis to be indexed.
They have not budged on letting un-publically editable wikis be indexed.
There is now a star requirement of about 500, and it appears to be steadily decreasing.
Many projects have reacted by moving or configuring their wikis to indexable platforms.

As a result, traffic to GitHub Wiki SEE has dropped off dramatically.

This is a good thing, as it means that GitHub is moving in the right direction.

I’m still going to keep the service up, as it’s still useful for wikis that are not yet indexed and there are still about 400,000 wikis that aren’t indexed.

Hopefully GitHub will continue to move in the right direction and allow all wikis to be indexed.

openpilot Replay Clipper

Wed, 12 Oct 2022 19:42:00 -0800

End to End Longitudinal Control is currently an “extremely alpha” feature in openpilot that is the gateway to future functionality such as stopping for traffic lights and stop signs.

Problem is, it’s hard to describe its current deficiencies without a video.

So I made a tool to help make it easier to share clips of this functionality with a view into what openpilot is seeing, and thinking.

GitHub repository

It’s a bit heavy in resource use though. I was thinking of making it into a web service but I simply do not have enough time. So I made instructions for others to run it on services like DigitalOcean, where it is cheap.

It is composed of a shell script and a Docker setup that fires up a bunch of processes and then kills them all when done.

Hopefully this leads to more interesting clips being shared, and more feedback on the models that comma.ai can use.

I later also ported this to Replicate here.

Datasette-lite sqlite-httpvfs experiment POC with the California Unclaimed Property database

Tue, 11 Oct 2022 20:36:00 -0800

There’s a web browser version of datasette called datasette-lite which runs on Python ported to WASM with Pyodide which can load SQLite databases. I grafted the enhanced lazyFile implementation from emscripten and then from this implementation to datasette-lite relatively recently as a curious test. Threw in a 18GB CSV from CA’s unclaimed property records here

https://www.sco.ca.gov/upd_download_property_records.html

into a FTS5 Sqlite Database which came out to about 28GB after processing:

POC, non-merging Log/Draft PR for the hack:

https://github.com/simonw/datasette-lite/pull/49

You can run queries through to datasette-lite if you URL hack into it and just get to the query dialog, browsing is kind of a dud at the moment since datasette runs a count(*) which downloads everything.

Elon Musk’s CA Unclaimed Property

Still, not bad for a $0.42/mo hostable cached CDN’d read-only database. It’s on Cloudflare R2, so there’s no BW costs.

Celebrity gawking is one thing, but the real, useful thing that this can do is search by address. If you aren’t sure of the names, such as people having multiple names or nicknames, you can search by address and get a list of properties at a location. This is one thing that the California Unclaimed Property site can’t do.

I am thinking of making this more proper when R2 introduces lifecycle rules to delete old dumps. I could automate the process of dumping with GitHub Actions but I would like R2 to handle cleanup.

openpilot Replay Cliper

Sat, 01 Oct 2022 00:00:00 +0000

Replicate: https://replicate.com/nelsonjchen/op-replay-clipper

https://github.com/nelsonjchen/op-replay-clipper/

📽 Capture short ~30 second route clips with the openpilot UI included, and the route, seconds marker branded into it.

Designed to try to encourage more interesting clips to be shared, especially with newer functionality such as “end to end longitudinal control”.

Originally a bit heavy, so instructions were made for others to run it on services like DigitalOcean/GitHub Codespaces. Since then, the clipper has been ported to Replicate which provides GPUs and a nice WebUI.

The UI rendering is composed of a shell script and a Docker setup that fires up a bunch of processes and then kills them all when done. On Replicate, it is wrapped by a Python script that handles the UI and non-UI processing of video and downloads.

Released Gargantuan Takeout Rocket

Sat, 16 Apr 2022 21:04:00 -0800

Finally released GTR or Gargantuan Takeout Rocket.

GitHub repository

Gargantuan Takeout Rocket (GTR) is a toolkit of guides and software to help you take out your data from Google Takeout and put it somewhere else safe easily, periodically, and fast to make it easy to do the right thing of backing up your Google account and related services such as your YouTube account or Google Photos periodically.

It took a lot of time to research and to workaround the issues I found in Azure.

I also needed to take apart browser extensions and implement my own browser extension to facilitate the transfers.

Cloudflare Workers was also used to work around issues with Azure’s API.

All this combined, I was able to takeout 1.25TB of data in 3 minutes.

Now I’m showing it around on Twitter, Discord, Reddit, and more to users who have used VPSes to backup Google Takeout or have expressed dismay at the lack of options for users who are looking to store archives on the cheap. The response from people who’ve opted to stay on Google has been good!

There is also a project page with additional details here.

Gargantuan Takeout Rocket

Sat, 16 Apr 2022 00:00:00 +0000

https://github.com/nelsonjchen/gtr

Used to do this with a VPS. It was still too slow to backup 1.25TB. TOOK HOURS! Built a Rocket.

It is comprised of a two major components:

A “GTR Proxy” Cloudflare Worker that HTTP3-izes the the Azure API and is capable of proxying base64 URLs to Google Takeout URLs.
A browser extension that coordinates all this and constructs a job plan to execute.

These two components are infinitely scaleable and can be used to backup Google Takeout at breathtaking, unprecedented speeds.

The GTR repo contains a guide on how to setup all of this.

A future revision or successor may target S3 or S3-like APIs. In particular, can we target Cloudflare’s R2 when it comes out?

Cell Shield

Wed, 28 Jul 2021 18:47:00 -0800

I built https://cellshield.info to scratch an itch. Why can’t I embed a spreadsheet cell?

You can click on that badge and see the sheet that is the data backing the badge.

It is useful for embedding a cell from an informal database that is a Google Spreadsheet atop of some README, wiki, forum post, or web page. Spreadsheets are still how many projects are managed and this can be used to embed a small preview of important or attention grabbing data.

It is a simple Go server that uses Google’s API to output JSON that https://shields.io can read. Mix it in with some simple VueJS generator front-end to take public spreadsheet URLS and tada.

I picked to base my implementation on shields.io’s as they make really pretty badges and have a lot of nice documentation and options.

I run it on Cloud Run to keep the costs low and availability high. It also utilizes Google’s implementation of authorization to get access to the Sheets API.

The badge generator web UI can also make BBCode for embedding (along with modern Markdown and so on).

I’ve already used it for this and that. The former outputs a running bounty total and the latter is community annotation progress.

A major Game Boy development contest is using it labels to display the prize pool amount: https://itch.io/jam/gbcompo21

Why I built this?

I built it for the Comma10K annotation project. I contributed the original badge which attempted to use a script that was used in the repo to analyze the completion progress. Unfortunately, this kept on diverging from the actual progress because the human elements kept changing the structure and organization and I had the badge removed. The only truth is what was on the spreadsheet.

Popularity

Compared to my other projects, I don’t think the popularity will be too good for this project though. I think it might have a chance to spread via word of mouth but it’s not everyday that a large public collaborative project is born with a Google Spreadsheet managing it. The name isn’t that great either as it probably conflicts with anti-wireless hysteria. Also, it’s not like there’s an agreed upon community of Google Spreadsheet users to share this knowledge with.

I also shamelessly plugged it here but I think the question asker might be deceased:

https://stackoverflow.com/questions/57962813/how-to-embed-a-single-google-sheets-cells-text-content-into-a-web-page/68505801#68505801

At least it was viewed 2,000 times over the last two years.

Maintenance

It’s on Cloud Run. The cost to keep it running is low as it sometimes isn’t even running and the Sheets V4 API it uses was recently declared “enterprise-stable”. Because of this, it will be super easy to maintain and cheap to run. I don’t expect any more than $2 a month.

GitHub Wiki SEE: Search Engine Enablement

Tue, 27 Jul 2021 18:12:00 -0800

I built this in March, but I figure I’ll write about this project better late than never. I supercharged it in June out of curiosity to see how it would perform. It’s just a ramble. I am not a good writer but I just wanted to write stuff down.

TLDR: I mirrored all of GitHub’s wiki with my own service to get the content on Google and other search engines.

If you’ve used a search engine to search up something that could appear on GitHub, you are using my service.

https://github-wiki-see.page/

Did you know that GitHub wikis aren’t search engine optimized? In fact, GitHub excluded their own wiki pages from the search results with their robots.txt. “robots.txt” is the mechanism that sites can use to indicate to search engines whether or not to index certain sections of the site. If you search on Google or Bing, you won’t find any results unless your search query terms are directly inside the URL.

The situation with search engines is currently this. For example, if you wanted to search for information relating to Nissan Leaf support of the openpilot self-driving car project in their wiki, you could search for “nissan leaf openpilot wiki”. Unfortunately, the search results would be empty and would contain no results. If you searched for “nissan openpilot wiki”, “https://github.com/commaai/openpilot/wiki/Nissan" would show up correctly because it has all the terms in it. The content of the GitHub wiki page is not used for returning good results.

It has been like this for at least 9 years. GitHub responded that they have declined to remove it from robots.txt as they believe it can be used for SEO abuse. I find this quite unbelievable. The mechanism to prevent this is to utilize rel="nofollow" on all user generated <a> links in the content and with newer guidance suggesting rel="nofollow ugc". By removing the content from the index, they have blinded a large portion of the web’s users to the content in GitHub wiki pages.

Until I made a pull request to add a notice to their documentation that wikis are not search engine optimized, there was absolutely no official documentation or notice about this decision. I still believe it is not enough as nothing in the main GitHub interface notes this limitation.

I believe there are many dedicated users and communities that are not aware of this issue who are diligently contributing to GitHub wikis. This really frustrated me. I was one of them. Many others and I contributed a lot to the [comma.ai openpilot wiki][op_wiki] and it was maddening that their work and my work were not indexed.

I noticed that a lot of technical questions had Stack Overflow mirrors rank highly and yet they weren’t Stack Overflow.

So what I did was this: I made a proxy site without a robots.txt that would mirror GitHub wiki content to a site that did not have a restrictive robots.txt. I put it up at https://github-wiki-see.rs. I called it GitHub Wiki SEE for GitHub Wiki Search Engine Enablement. It’s kind of a twisted wording of SEO as I see it as enabling and not even “optimizing”.

What’s the technology behind it?

The current iteration is a Rust Actix Web application. It parses the URL and makes requests out to GitHub to get the content. It then scrapes the content for relevant HTML and reformats it to be more easily crawl-able. It then serves the content to the search engine and users who come in from a search engine with a link to the project page and reasoning at the bottom as a static pinned element. Users aren’t meant to read that page and are only meant to browse to the original GitHub page.

With these choices, I could meet these requirements:

Hosted it on Google Cloud Run just in case it doesn’t become popular.
Made it easy to deploy.
Made it cheap.
Made it small to run on Cloud Run.
Made it low maintenance.

Another considering was that it needed to not get rate limited by GitHub. To do this, I made the server return an error and quit itself on Cloud Run if it detected a rate limiting response to cycle to a new server with a new client IP address. Unfortunately, this limited the number of requests each server could handle at a time to a minimum of one.

Supercharging it

I originally built the project in March. I monitored the Google Search Console like a hawk to see how the openpilot stuff I had contributed to was responding to the proxy setup. Admittedly, it got a few users to click through here and there and I was pretty happy. For a month or two, I was pretty satisfied. However, I kept on thinking about the rest of the users who lamented this situation. While some did take my offer to post a link somewhere on their blog, README, or tweet, the adoption wasn’t as much as I had hoped.

I had learned about GH Archive which is a project that archived every event of GitHub since about 2012. I also learned that the project also mirrors its archive on BigQuery. BigQuery allows you to search large datasets with SQL. Using BigQuery, I was able to determine that about 2,500 GitHub Wiki pages get edited every day. In a week, about 17,000. A month? 66,000 pages. My furor at the blocking of indexing increased.

I decided to throw $45 at the issue and run a big mega-query and queried all of GH Archive. Over the lifetime of GH Archive, 4 million pages were edited. I dumped this into a queue and processed it with Google Cloud Functions to run the HEAD query on all the pages in the list and re-exported it to BigQuery. 2 million pages were still there.

You can find the results of this in these public tables on BigQuery:

github-wiki-see:show.checked_touched_wiki_pages_upto_202106
- Checked with HEAD
github-wiki-see:show.touched_wiki_pages_upto_202106

With this checked_touched_wiki_pages_upto_202106 table, I generated 50MB of compressed XML sitemaps for the GitHub Wiki SEE pages. I called them seed sitemaps. The source for this can be found in the sitemap generator repo.

BigQuery is expensive, so I made another sitemap series for continuous updates. An earlier version used BigQuery, but the new version runs a daily GitHub Actions job that downloads the last week of events from GH Archive, processes them for unique Wiki URLs and exports them to a sitemap. This handles any ongoing GitHub Wiki changes. The source for this can also be found in the sitemap generator repo.

The results

I am getting thousands of clicks and millions of impressions every day. GitHub Wiki SEE results are never at the top and average a position of about 29 or the third page. The mirrored may not rank high and I don’t care as the original content was never going to rank high anyway so the content appearing at all in the search results of thousands of users even if it was sort of deep satisfies me.

All in all, a pretty fun project. I am really happy with the results.

My contributions to the openpilot wiki are now indexed.
Everyone else’s contributions to their wikis are now indexed as well.
I learned a few things or two about writing a Rust web app.
I learned a lot about SEO
I feel I’ve helped a lot of people who need to find a diamond in the rough and the diamond isn’t in the URL.
I feel a little warm that I have a high traffic site.

I have no intention of putting ads on the site. It costs me about $25 a month to run but for the high traffic it gets, the peace of mind I’ve acquired, and the good feelings of helping everybody searching Google, it is worth it.

I did add decommissioning steps. If it comes to it, I’ll make the site just directly redirect but after GitHub takes Wikis off their robots.txt.

Things I Learned

This is a grab-bag of stuff I learned along the way.

Sitemaps are a great way to get search engine traffic to your site.
Large, multi-million page sitemaps can take many months to process. Consider yourself lucky if it parses 50,000 URLs a day.
- The Google Search Console saying “failed to retrieve” is a lie. It means to say “processing” but this is apparently a long standing stupid bug.
GoogleBot is much more aggressive than Bing.
- Google’s Index of my proxy site is probably a lot larger than Bing’s by a magnitude or two. For every 30, 40, or even 100 GoogleBot requests, I get one Bing request.
- This is important for serving DuckDuckGo users as it is their index.
The Rust Tokio situation is a bit of a pain as I am stuck on a version of Tokio that is needed for stable Actix-Web. As a result, I cannot use modern Tokio stuff that modern Rust crates or libraries use. I may port it to Rocket or something in the future.
A lot of people search for certification answers and test answers on GitHub Wikis.
GitHub Wikis and GitHub are no exception to Rule 34.
Cloud Run is great for getting many IP addresses to use and to swap out if any die. The outgoing IP address is extremely ephemeral and changing it is a simple matter of simply exiting if a rate limit is detected.
- This might not always work. Just exit again if so.
GCP’s UI is nicer for manual use compared to AWS.
The built-in logging capabilities and sink of GCP products are also really nice. I can easily re-export and/or stream logs to BigQuery or to a Pub Sub topic for later analysis. I can also just use the viewer on the UI to see what is going on.
I get fairly free monitoring and graphing from the GCP ecosystem. The default graphs are very useful.
Rust web apps are really expensive to compile on Google Cloud Build. I had to use 32-core machines to get it to compile in underneath the default timeout of 10 minutes. I eventually sped it up and made it cheaper to build by building on a pre-built image along with a smaller instance but it is still quite expensive.
The traffic from the aggressive GoogleBot is impressively large. Thankfully, I do not have to pay the bandwidth costs.
My sloppy use of the HTML Parser may be causing errors but RAM is fairly generous. It’s Rust too so I don’t really have to worry that much about garbage collection or anything similar.
Google actually follows links it sees in the pages. When I started, it did not. Maybe it does it only on more popular pages.
The Rust code is really sloppy, but it still works.

GitHub Wiki SEE

Thu, 01 Apr 2021 00:00:00 +0000

https://github-wiki-see.page/

A project to get GitHub Wikis indexed by Google Search.

Explanation is right on the front page.

If a search engine can’t see it, it may as well be invisible.

Long-term project involving Rust, BigQuery, Cloudflare Workers, and lots of random hosting.

Probably one more the more visited sites on the internet and a continous effort to keep costs low.

Has caused GitHub to revisit their policy on a limited subset of Wikis meeting some criteria.

Will continue until all Wikis are indexed even if they are publically editable or don’t meet some star count criteria.

Personally, I just got really dismayed all this documentation I was doing on a Wiki wasn’t visible to Google. Then I realized it was a good idea to make it visible. Then I realized others don’t realize it was invisible at all. This was a problem.

The short lived life of Breakout Bots for Zoom Meetings

Sat, 12 Sep 2020 19:42:00 -0800

With the pandemic happening, it’s been tough for many organizations to adapt. We’re all supposed to be together! One way organizations have been staying together is by using Zoom Meetings.

At work, we’ve been using a knock-off reseller version called RingCentral Meetings. From looking at their competitors, Zoom has pulled out all the stops to make their meeting experience the most efficient, resiliant, rather cheapest, and reliable experience.

One of these features is “Breakout Rooms”. With breakout rooms, users can subdivide their meetings to make mini-meetings from a bigger meeting. For most of the year, there has been an odd restriction on Breakout Meetings though.

You cannot go to another breakout room as an attendee without the host reassigning you unless you are a Host or a Co-Host.

Obviously, this can result in a lot of load upon the poor user who is desginated the host. Even the Co-Hosts can’t assign users to another breakout room.

So I made a bot:

https://github.com/nelsonjchen/BreakoutRoomsBotForZoomMeetings

It’s quite a hack, but it basically controls a web client that is a Host and assigns users based on chat commands and on attendees renaming themselves.

Last night, I just finished finally optimizing the bot. It should be able to handle hundreds of users and piles of chat commands.

This morning I learned that Zoom will add native support for switching:

https://www.reddit.com/r/Zoom/comments/irkm82/selfselect_breakout_room/

It was fun, but as brief as the bot existed, the main purposes are numbered. Maybe someone can reuse the code to make other Zoom Meeting add-on/bots?

Technical and Learnings

I knew about this issue for many months but I never made a bot because there wasn’t an API. I waited until I was very disatisified with the situation and then made the bot. I should have just done it, without the API.
Thankfully, by doing this quickly, I only really spent like a week of time on this total. With some of the tooling I used, I didn’t have to spend too long debugging the issues.
The original implementation of this tool was a copy and paste into Chrome’s Console. This is a much worse experience for users than a Chrome extension. Kyle Husmann contributed a fork that extensionized the bot and it is totally something I’ll be stealing for future hacks like this.
Zoom got to where they are by doing things their competitors did not and rather quickly. This feature/feedback response is great and is why they’ve been crushing. I feel this is a feature that would have taken their competitors possibly a year to implement. At this time, Microsoft Teams is also looking to implement Breakout Rooms.
Zoom’s web client uses React+Redux. They also use the Redux-Thunk middleware. They do stick some side-effects in places where things shouldn’t but whatever, it works. It did make integration very hard if not impossible at this layer for my bot. However, subscribing to state changes was and is very reliable.
RxJS is a nice implementation of the Reactive Pattern with Observables and stuff like with RX.NET. Without this, I could not fit the bot in so few lines of code with the reliability and usability that it currently has.
With RxJS and Redux’s store, I was able to subscribe to internal Redux state changes and transform them into streams of events. For example, the bot transform the streams of renames and chat message requests into a common structure that it then merges to be processed by later operators. Lots of code is shared and I’m able to rearrange and transform at will.
The local override functionality is great if you want to inject Redux Web Tools into an existing minified application. If you beautify the code, you can inject dev tools too. Here’s the line of code I replaced to inject Redux Dev Tools into Zoom’s web client.

Previous:
```
, s = Object(r.compose)(Object(r.applyMiddleware)(i.a));
```
After:
```
, s = window.__REDUX_DEVTOOLS_EXTENSION_COMPOSE__(Object(r.applyMiddleware)(i.a));
```
This was super helpful in seeing how the Redux state changes when performing actions in a nice GUI.
fuzzysort is great! I’m quite surprised at how unsurprising the results can be when looking up rooms by a partial or cut up name.
The underlying websocket connection is available globally as commandSocket. If you observe the commands, they are just JSON and you can inject your own commands to control the meeting programatically.
Since Breakout Room status is sent back via WebSocket, the UI will automatically update. I did not have to go through the Redux store to update the Breakout Room UI.
UI Automation of Breakout Room UI: ~100ms vs webSocket command: 2ms. Wow!
Chat messages are encrypted and decrypted client-side before going out on the Websocket interface. This probably doesn’t stop Zoom from reading the chat messages since they are the ones who gave you the keys. I could have reimplemented the encryption in the bot but after running some replay attacks, the UI did not update. Without the UI updating, I didn’t feel it was safe for the host to not know what the bot sent out on their behalf.
Even if I had implemented the chat message encryption, the existing UI automation of chat messages hovered around 30ms. The profiler also showed that the overhead of automation wasn’t that much and much of the chat message encryption contributed to the 30ms. It wouldn’t have been much of a performance win if I implemented it myself.
Possible Security-ish Issue - Since it isn’t an acceptable bounty issue, and hosts can simply disable chat, I will disclose is possible to DOS the native client by simply spamming chat. If there is too much text, the native clients will lock up and require a restart. Native clients don’t drop old chat logs. Web clients don’t have this issue as they will drop old chat automatically. Last but not least, it is possible for the Host to disable chat in many ways to alleviate such an attack. I wasn’t sure if to let my load_test bot out there or not because of this but after reflection, I think it’s OK.
RxJS is MVP. Or rather the Reactive stuff is MVP. If you can get it into a stream, it’s super powerful. Best yet, you can reuse your experience from other ecosystems. I was previously using Rx.NET for some stuff at work. I’ll be definitely looking at using the Python version of this in the future.
- Though it did seem the differing Reactive stuff have different operators. The baselines are still great though and you can always port operators from one to another implementation.
Rx is really helped if you can use TypeScript. Unfortuntately, I never got to this stage.

That said, this was fun! Now onward to the next thing.

Reverse RDP into Windows on GitHub Actions

Sat, 07 Dec 2019 17:24:55 +0000

Ever wonder what the Desktop of the Windows Runners on GitHub Actions looks like?

Or perhaps you’re missing the ability to RDP into build agents like on Appveyor.

I wrote some steps that use ngrok to make a reverse tunnel possible. I also turned on RDP if it wasn’t on already and set a password.

Take a look here!

https://github.com/nelsonjchen/reverse-rdp-windows-github-actions

Stop Sims 2 Purple Soup or Pink Flashing Crashes on Windows 10 and Modern Hardware with D9VK

Wed, 02 Oct 2019 06:08:55 +0000

Wrote a guide on how to fix the Purple Soup or Pink Flashing Crashes for The Sims 2 on Windows 10 1903+ on powerful gaming computers. It’s great to make this accessible again to gamers with powerful gaming computers.

Guide Link: https://docs.google.com/document/d/19JMr-FQSU3AlF7Kyvcrr7awTiHBDQSUNLG6qfaI6rOs/edit#

General idea: Replace Microsoft DirectX 9 with D9VK.

D9VK is actually actively developed and fixes bugs. It also translates graphics to a stack that is more open and stable.

So this:

Turns to this:

Fast and lightweight headless Qt Installer from Qt Mirrors: aqtinstall

Sun, 02 Jun 2019 22:02:00 -0800

As part of my work on the Barrier project, we needed a way to build it with the [Qt installer]from an online CI/CD service. On some platforms, such as Appveyor, Qt may be preinstalled. But on some other platforms such as Azure Pipelines, Qt may not be pre-installed. Nor on Travis. Nor on CircleCI. For Linux and macOS, there is a saving grace. The Qt libraries can be installed headlessly from their operating system repositories such as apt or brew. But this luxury does not exist as well on Windows. While Chocolatey does offer some Qt packages, they tend to ancient or non-official.

There is a toolkit called CuteCI that let you use the Qt Installer and script the GUI headlessly. It’s janky and big though. Additionally, to pin versions, you must download 3 to 5 GB of Qt Installer to install a pinned version of Qt.

So, I found this: https://lnj.gitlab.io/post/qli-installer/

Apparently it’s possible to mimic the Qt Installer from a Python script.

To save it from possible bit-rot, I forked it to GitHub. I added parallel downloading, Windows support, and a CI system. I pushed Azure-Pipelines and each commit could cause 100 jobs to run. Across all the various platforms. Azure Pipelines did not break a sweat.

CuteCI is still extremely useful if you must use the official installer with official support. But the speeds with my fork were insane. It would normally take many minutes to install Qt with CuteCI. My QLI-Installer fork? 10 or 20 seconds.

But it turns out there’s an even better installer than mine!

https://github.com/miurahr/aqtinstall/

aqtinstall

Classy reusable Python structure
On PyPi. You can do pip install aqtinstall and install aqtinstall for immediate use.

This is also another fork of qli-installer. The CI system was a in a bit of a sorry state though. So I contributed matrix generated jobs to test aqtinstall across Mac, Windows, and Linux. @miurahr was able to dial down the installation and testing madness too and ramp it up in some places. Actual Qt applications are built to test if aqtinstall work. Additionally, there was the insight that mobile platforms generally always require the latest Qt version.

So, if you want a fast and lightweight headless Qt installer that installs from Qt Mirrors and is continually tested, use aqtinstall!

Non-encumbered Windows Bonjour SDK DLLs and Libs

Sat, 01 Jun 2019 19:15:00 -0800

This was one of those adventures where you find a problem and it snowballs from there.

As part of my work on the Barrier project, we needed a way to build it with the Bonjour SDK from an online CI/CD service. Bonjour is used by Barrier to discover other barrier hosts on the same network. On Windows, the SDK files are needed. Other platforms natively come with the library or similar APIs already preinstalled.

But there’s an issue. The SDK is behind a login wall on Apple’s site. Rehosting it might be possible but it might violate some agreement within the installer. Luckily, the sources are open and are under very permissive licenses.

To save some work, I reused the XBMC project’s fork and made my fork of the SDK. I didn’t change any of XBMC’s changes, if any. I added a CI system to build the necessary libraries in the same manner that Apple did.

So, if you need a free, Cloud CI system accessible Apple Windows Bonjour SDK, look at the releases in my repo! The library files are there and users can fork my build to build the stubs their own way as well.

Barrier CI System WIP

Thu, 30 May 2019 22:02:00 -0800

This was one of those adventures where you find a problem and it snowballs from there.

I wanted to use barrier which is a tool for sharing mouses between computers. It is a fork of Synergy, which had gone commercial.

So, I noticed on their GitHub repo that the builds for various platforms was inconsistent. OK, maybe I can help fix their CI systems.

More worringly, there was some Appveyor CI system configured. It was not configured with the appveyor.yml method and the build process was non-transparent. It was also failing.

My current CI system of choice for open source projects is Microsoft’s Azure Pipelines. For open source projects on GitHub, they’ll provide 10 parallel jobs of Mac, Linux, and Windows. Mac and Windows usually only show up as premium options or are severely rate-limited. As far as I can tell, whatever cluster Microsoft has created for their service, it’s really not limited by capacity.

The good news is that they were fans of it and I was able to move them. It’s still a work in progress. The work done here has draw in some general “improve the ecosystem” enhancements which I will elaborate on in future posts.

An Android Accessibility Service for Eventbrite Organizers: PassShout

Fri, 15 Mar 2019 19:15:00 -0800

The Scene:

It’s Saturday. And it’s like so many people have free time on that date.
You’re manning a convention center ticket check-in booth.
Your eyesight just isn’t the best. After looking at hundreds, if not thousands of them, it can be a blur.
You’re using Android.
There’s a long line and people have different pass or ticket types, sometimes even if they are in the same group.
You’re processing thousands of people.
You might not even have a table.
You’re using Eventbrite Organizer.

I have a tool for you! It’s called PassShout.

In addition to a chime, it’ll read out loud the pass/ticket type.

This greatly reduces operational mistakes. PassShout is part of an implementation of a method in occupational safety called Pointing and Calling. PAC has been adopted widely thoughout Asia and the NYC Metro to reduce incidents on trains systems. By calling out the type, things move faster and mistakes are reduced.

Implementation

This is the first Android app I’ve written in Kotlin. It’s amazing what the Android Accessibility API offers for third party programs to make things more accessible. Unfortunately, there does not seem to be an equivalent for iOS.

Kotlin is really easy to read and write. I can see how it’s not Scala and is closer to a usable modern Java. It’s basically the Swift of Android.

NC QEMU

Tue, 25 Dec 2018 00:00:00 +0000

Changes

Azure Pipeline to build x86_x64 target QEMU in MSYS2 on Windows
Upgraded to Capstone with fixes that allow building in MSYS2.

About

“NC QEMU” is a very lightly augmented and experimental fork of QEMU to build on Azure Pipelines and with Windows Hypervisor Platform (WHPX) support for users who care the most about running the x86_64 target fast. This build only cares about the x86_64 target and other targets are not built.

The output is a zip file with some DLLs and qemu-system-x86_64.exe. This QEMU distribution can be run from a folder by itself. Alternatively, qemu-system-x86_64.exe can be dropped into QEMU for Window’s installation directory at C:\Program Files\qemu, replacing the existing version. Some features like USB network redirection might be missing though.

The pipeline provides a differently compiled QEMU compared to QEMU for Windows’s own build instructions which are cross-compiled on Linux. Notably, NC QEMU is built on a Microsoft-provided copy of Windows in MSYS2 with access to the Windows SDK headers for WHPX distributed by Microsoft. The MingW64 Toolkit which QEMU for Windows is built with has unfortunately currently not reproduced the WHPX headers in a free software manner. Users who want a QEMU for Windows with WHPX support but aren’t licensed nor want to accept the terms of installing the Windows 10 SDK are out of luck.

Another benefit is that this project provides a declaratively made and executable pipeline that builds QEMU for Windows. Fork, setup an Azure Pipeline, and adjust if needed. This can be considered a bit “executable documentation” for building a QEMU x86_64 target of this nature. The build logs are public and can be used as reference. For users having trouble building or configuring their systems to build QEMU, this reproducible setup can be quite useful.

Why QEMU+WHPX for `x86_64`

Absolutely not exhaustive:

Acceleration is great. You can reach near-native speeds with acceleration.
WHPX is native to the OS. No foreign kernel drivers/modules/extensions needed.
It’s great for users who don’t have access to Intel’s HAX because they either want Hyper-V and/or AMD support. With AMD CPUs being multi-core bargain monsters, developers and power users on AMD are becoming more numerous.
QEMU+WHPX boots Windows ISOs. HAX currently does not.
QEMU is easier to hack on, script, and developed by many organizations, not one. WHPX support was contributed by Microsoft but is probably most useful for Google’s QEMU-based Android emulators!
Users hacking on or working with QEMU in Windows can bring their work to Linux KVM accelerated systems easier. And vice-versa.

Packer WHPX workaround

The executables provided here do work for Packer. Simply add the directory with qemu-system-x86_64.exe to the PATH. Packer does not currently recognize whpx as a valid argument for the acceleration key. The workaround is to add it to qemu-args as a manual argument.

Additionally, the cpu argument of QEMU does not support cpu=host for whpx. Specify something supported manually.

References

Orbital (PS4 Emulator) Build instructions: https://github.com/AlexAltea/orbital/wiki/Building

Gooey and PyInstaller for easy to write, distribute, and use scripts!

Sat, 15 Dec 2018 20:36:00 -0800

What is:

Easy to write
Easy to distribute and portable
Super Powerful
Cross Platform
Puts computer novices and professionals on the same level?

I think it’s a combination of Gooey and PyInstaller. If you put those two together, you get all those above.

It’s Python, so it’s easy to write. There are so many guides online and a wide assortment of libraries. Most importantly, you get libraries that have a strong eye towards ergonomics. Python is a wonderful beginner’s language that some people even call an executable psuedolanguage.
PyInstaller bundles applications into a single folder or exe so it’s easy to distribute.
You get the full force of Python’s language and its ecosystem behind you.
Python is cross platform, but Gooey’s GUI is also as well. Don’t be expecting to make “Destiny”-style GUIs with it. But do be expecting a surprising amount of native-ness.
While PyInstaller does package the application up into a portable Python script, the existing command line invokation is still left for professionals. Simply add --ignore-gooey to the list of arguments. Now it’s just a plain old script.

The Gooey PyInstaller Demo was presented at a DesertPy meetup in August of 2018. It’s a project showing the development of a simple argument parsing script all the way up to a portable GUI. ~~Future improvements to the demo include some CI scripts and configurations.~~ This was completed on 2018-12-15.

Demo Project Link: https://github.com/nelsonjchen/gooey-pyinstaller-demo

Switched to Hugo

Sun, 29 Nov 2015 00:37:59 -0800

Wow! I think it has been nearly two years since I last updated this blog. I have decided to change the blog a bit and port it over to hugo which is a static site generator written in Go. It is very fast compared to nanoc which I was using. I do not know if I quite agree that this is just as customizable as nanoc but it is clearly faster. I was not taking too much advantage of the previous Foundation based flexibility of the nanoc site and I believe it may had become a burden. Also, bitrot was beginning to seriously set in. Luckily, it is a static site but it does make some tears shed when you have to pin some really old gems and use a really old version of Ruby to build and deploy the site. Unless the kernel syscalls change dramatically, I will probably be able to compile with this version of hugo indefinitely.

Anyway, the nanoc versions troubles me no more. I am using a hugo version of the site now. As an improvement, I have also setup automatic deployment of my master branch to Google’s storage and configured CloudFlare to provide TLS and all the goodies that come with that. I guess I could also start using something like prose.io to write my posts as well. I doubt I will go that far! But, it is very nice to know that is possible.

I am now just using a theme called hyde-y which is off-the-shelf. Of course, I did customize it a little bit. hugo’s overriding system is plain and simple to use.

Why would anyone want summaries on the front page? Just give the complete content.
The titles of the posts on the front page should have the same information.

I addressed all that, and while it was a bit of a hunt, it was fairly clear what needed to be done.

I do not think I will go the full mile with customizing hugo like I did with nanoc. The complexity was hurting more than it helped. I guess I have switched over to being more minimalist since I concieved the original site. Foundation is great if you need the kitchen sink and everything but it is really overkill. It was also another maintenance burden with its rather relentless upgrade schedule. I am taking it slow from here on out.

Running Windows XP using Libvirt inside Ubuntu inside VMWare Fusion

Mon, 06 Jan 2014 18:09:02 -0700

So I’m investigating running Windows XP inside libvirt in Ubuntu on my MacBook for continuous integration testing purposes with Jenkins. I tried for many hours to get it working but Windows would not boot up or it would get stuck on the NTLDR cannot be found issue. As it turns out, I did not enable “Enable hypervisor applications in this virtual machine”. I was under the assumption that I would not need to check that box as I wrongly assumed qemu would handle all the needed machine translation and that an error with bootup is not the fault of lacking the ability to use a CPU feature to emulate something or the disk emulation being borked. With five different ISOs of various pedigrees, I tried installing Windows XP with virt-manager and all of them failed to bootup with various disk image formats like qcow2 or raw. It was only until I checked that checkbox did it then work.

That’s my surprise for today. I really wished I checked that checkbox earlier! It determines if your machine can bootup and only using qemu without kvm will not cut it for booting Windows XP inside libvirt inside Ubuntu inside VMware Fusion.

BeamNG DRIVE on Mac OS X with CrossOver

Sat, 03 Aug 2013 16:06:37 -0700

UPDATE 2015-11-28: Hey BeamNG forum users. These instructions worked for me way back when I was using CrossOver. You might be able to get similar or better results with Wineskin Winery or some other Wine wrapper. And those are free too! I don’t run BeamNG on my local laptop anymore nowadays but I Game on Amazon EC2 and it works fairly well with BeamNG at a very minimal cost.

Gotta make this quick. This post will change.

Here’s how to get BeamNG Drive working in Mac OS X. I’m sure there are equivilent instructions for Wineskin but I have no time to find out.

Make a new bottle in Crossover. Make sure it is a Windows Vista bottle. Run the Wine configuration setting inside it and set the version to Windows 7 in one of the sheets. Enable the Mac Driver so you don’t run it through X11.

Install the Modern DirectX runtime into it. Install BeamNG into it. Mute your speakers. Change the audio options to use OpenAL for output. Unmute speakers. Enjoy.

So far this works for the tech demo. I’m going to try it on Alpha later.

UPDATE: It works on Alpha. Yay!

Mount USB devices in Virtualbox-based Vagrant

Wed, 24 Jul 2013 17:03:05 -0700

If you want to mount USB in Virtualbox, you have to do this solution. At the time of this post, a typical Google search for this would go to a solution at advocated using the attachusb command in VboxManage in the Vagrantfile provider customization section. This will not work because the VM is off at the time of bootup. It looks like the original author of this gist did not repost his solution back to the thread.

The solution, in other words, is to use USB filters to automatically connect devices to your VM. USB filters can be added while the machine is powered off. They’ll be applied upon boot or, in this case, vagrant up.

My current vagrant setup

Sat, 22 Jun 2013 20:09:42 -0700

I firmly believe that Vagrant is the quickest way from nothing to a running and preconfigured development environment on any machine and especially Macs. For me, the ‘works on my machine’ problem is the biggest reason I run Vagrant. Day-to-day though, Vagrant is probably the easiest to use UI for Virtualbox. If my work actually had money to give me for the VMWare plugin, I believe it would be a better UI for VMWare Fusion as well.

The Basics

This is enough to get started with Vagrant and to reap the rewards.

That’s great and all, but these are the basics. At the very least, you’ll be able to bring up some boxes that don’t require special plugins up.

Frills

You don’t need these but I do! I usually build my Vagrant boxes with Opscode Chef, a configuration management system. For reference, a Chef cookbook is a series of statements about how a machine should be setup.

Most of these frills are plugins. To build the boxes I make, you’ll usually have to install or use these.

Berkshelf is a dependency resolution manager for Chef. I use Berkshelf as a gem along with the corresponding vagrant plugin. With this, when I run berks cookbook <name>, I can make a Virtual Machine that can be created and destroyed quickly from scratch for whatever purpose. I could do vagrant init but berks cookbook has it beat by creating a directory structure that’s pretty much a Chef cookbook. Even if I don’t intend to redistribute said cookbook, the VM made is perfectly fine for tryout purposes.
I like using the Opscode Bento boxes. They are minimal and they have already been uploaded to S3 on Opscode’s dime. In a Vagrantfile, you can set the box URL for a Vagrant basebox to be downloaded. These boxes are great.
You can’t use the Opscode Bento boxes without the Vagrant Omnibus plugin. Those boxes do not include Chef so you must install Chef at runtime.
Just so you don’t get warnings about the Virtualbox additions being out of date, there’s a Vagrant plugin to automatically update the guest additions if needed. This one is really optional and it’s use just surpresses that warning you get if you bring up a vagrant box with old guest additions.

The Future

In the future, I’ll like to be able to test my boxes to make sure they stay working as the things they pull from the internet change. For this, there’s Test Kitchen.

Unfortunately, it’s still really cutting edge. However, there are guard plugins and this Youtube Video and blog post. That video is very much a must see for anybody who appreciates TDD.

Full integration testing on your own laptop is very attractive to me. I keep mine plugged in and I find it disturbing that the rest of the cores on this MacBook Pro just lie cool.

And also, maybe if I get some cash, I might drop some money on the VMWare plugin and VMWare Fusion. If I want to simulate multiple servers at a time and heavier loads, it would make that much faster.

A Windows "QuickStart"

Sun, 26 May 2013 20:33:11 -0700

I paid Swish $25 and $8 shipping for this two months ago. I think I had my share donated to the Khan Academy.

It’s the “Windows Quickstart Kit for Mac Developers”. It includes:

A full Windows 8 Pro license
A Parallels Desktop license
A USB Stick with an ISO on it with a real Microsoft Certificate of Authenticity. Genuine Windows!

I’m not sure why Microsoft saw fit to spend two months packaging a “quickstart”. Why Microsoft did not see fit to provide this quickstart through digital distribution is beyond me. They could have emailed a link to some keys and ISOs and maybe even let a distributor like Digital River do this. Instead, they saw fit to contract this work out to a physical media distribution company. It seems with these charity-ware deals that the best way to donate money is to allocate a large chunk to some physical distributor.

I’m so stoked to try and develop for Windows Phone. I’m going to virtualize Windows 8 and develop a native app for a platform on which I have no way to run natively. I’m also going to jump over some chairs in celebration.

Nuh uh, virtualization is cheating and is really anti-dogfood. If I have to boot up a RAM hungry OS to RUN the app, it’s a failure. If I have to develop the app inside a virtualized OS, that would be even more of a failure since I would have to bear with the performance and non-nativeness penalty. There’s no way a good app can come out with so many barriers like that. Google realized that its Eclipse toolkit was a problem and switched to IntelliJ IDEA to reduce their barriers and gain an awesome IDE in the meantime. I highly doubt Microsoft will provide a free cross-platform and light SDK. The technical barriers are just too high. It’s totally not like their Xbox where game developers are used to putting up with shit like that. Mobile developers are in general more finicky and prefer native tools. I’m not sure if this is still the case now but the Android team saw fit at one point to maintain the Android compilation toolchain for the entire OS on OS X. You can compile a Linux system on OS X!

With that said, this cool cardboard box is all that remains. It would be well designed if it never existed. It’s way too late for that though.

It’s made up of two big pieces. After ripping off the plastic wrapper, there’s a windows logo cutout sleeve thing to keep the box with a flap closed. It’s a pretty retail package for something that is only sold online.

When you open the box, you have a welcome card with some very basic instructions on the back on the left and some coupon card things on the right.

Beneath the welcome card is a USB drive. It has a Microsoft certificate of authenticity on it, so you know its real and has real Microsoft binary bits. There’s also a tracking ID on it too. The Microsoft logo is embedded in plastic on the back and not some cheap printer job. It’s certainly done with style to make the USB drive seem to be actually worth something.

On the USB Drive is just an ISO. There are no other files The drive itself isn’t bootable. It is formatted with NTFS though. At the very least, it’s readable on all platforms. It also enumerates as a device of the name Windows 8 Pro in system profiler. Quite a Matroska doll. I guess having an ISO is easier than mounting a USB drive in a virtual machine in terms of instructional material.

The cards on the right had keys on them. They’re business card sized and feel like them too.

On the back were some keys. Why they felt the need to kill all these trees and grow soy beans to produce the ink for printing on said dead tree material to wrap around these numbers is beyond me.

It took two months for these two alphanumeric strings to get here. The lack of timeliness on delivering these two codes and an ISO does not look well for Microsoft’s Windows efforts at all. It would take 30 minutes to illegally obtain these materials. 30 minutes is far less than two months. Oh, I almost forgot, they don’t even include Parallels in the package. You will have to go download that from Parallel’s site. What a “quickstart” indeed.

Here's a compiled version of Keycastr.

Wed, 22 May 2013 13:02:12 -0700

Update: Keycastr is currently maintained at github.com/keycastr/keycastr. Please use that repository first.

I just compiled a version of keycastr and put it here since I could not find a precompiled version that’s relatively new. The release artifacts are available here:

Keycastr releases

Keycastr is used for presentational demos. It’s free and the source is here in this GitHub repo. You will need a modern version of Xcode to compile it.

The version that is compiled is by creemama. I only blindingly updated the settings with Xcode recommendations. This “works for me” on OS X 10.8.3.

Boot Camp, Windows 7, and USB Installation Gotchas

Mon, 13 May 2013 15:23:28 -0700

A few days ago, I decided to pull the trigger and install Windows on my Retina MacBook Pro at a LAN Party. It’s a perfectly fine machine with a decent graphics card and CPU. I started Boot Camp Assistant and followed through with the instructions to install from the USB drive using the ISO of Windows 7 I had on my Desktop. If you follow Apple’s steps to the letter, your installation of Windows will never proceed.

If you do follow the steps, you will most certainly get an error of “Setup was unable to create a new system partition or locate an existing system partition. " when you select the BootCamp partition that you had just formatted in the Windows installer on your Mac. This can occur on PCs too but basically the issue is that the USB drive is initialized as another valid bootable disk. If you were to Google this, you will find solutions online that involve yanking out the USB drive and reinserting it before pressing “Install Now” on the Windows 7 installation screen. This “Install Now” screen is a big centered button that prompts your to install now and the arrow is set inside a blue circle that’s like a gem.

This is where Apple’s helpfulness can get in your way. By default, Boot Camp Assistant copies the Boot Camp support files to your USB drive. It also adds an Autounattend.xml file to the root of your drive to automatically install Boot Camp support drivers and utilities in Windows. With Autounattend.xml present, you will not have the opportunity to yank out and reinsert the USB drive before continuing. Instead, the setup just proceeds as if that button was pressed. While this is a good file for disc installations, it causes a massive problem for USB installation which is most likely to be the case for all new Macs from now on since they do not include disc drives by default. If you try to ask Apple to ‘fix’ this, they will just blame Microsoft which admittedly isn’t blame free either.

To solve this problem, move or delete Autounattend.xml on the root of the USB drive and proceed with the normal steps afterwards . After Windows is done installing and since Autounattend.xml was nullified, you will have to run the Boot Camp support installer from the BootCamp folder on the USB drive.

It’s a shame Apple’s instructions just do not work. The fix required would probably mean removing Autounattend.xml functionality for USB Drive installations. In the meantime, this worked for me.

Reddit Link: LEARN VERSION CONTROL if you are in UCSB CS!

Tue, 07 May 2013 16:19:28 -0700

Here’s a comment I wrote on /r/UCSantaBarbara a month ago. Learning Git is a very valuable tool for any aspiring or lazy programmer.

UCSB CS FAQ comment thread

At my current place of somewhat employment, the candidates for our internship offering job did not know how to use Git. I hope my post will have changed something.

If you’re new to Git, git-scm.com is still a good starting point.

And suddenly, nearly a decade later, a blog appears! Again.

Sun, 14 Apr 2013 22:05:53 -0700

I’m ashamed to say I’ve neglected this domain and site for 9 years. In that time I’ve gone through high school and university in that time. In each period, I’ve attempted putting up blogs, forums, another blog, and other similar nonsense. I did not follow through with whatever resolutions I may have made to keep the site maintained or write down the latest results from my last screwing around with my projects. Basically, for the past 9 years, I don’t really have much to show for it other than being able to rattle off some helpful anecdote from experiences I have never written down here and there.

Remember kids, the only difference between screwing around and Science is writing it down.

– Adam Savage, Video

I thought I was conducting Science for the past few years; in reality, I’ve only been really screwing around.

Most of my experiments in the past few years have also ended in spectacular failures. I never wrote about my failures even during the times I had a blog available. This is where my fails turned into an epic fail. I recently watched a TED talk on this and learned that this phenomenon is called research bias. I should not have withheld what I had learned from failing. My memory of my failures only fade away until someone mentioned it and I rattle off some anecdote. This obviously won’t scale and does not do much to help me demonstrate any skills when I can’t even get my foot in the door.

I’m going to change that with the launch of my new site. I’ll be honest here, this is very much a career move. However, it is also a personal one for reasons I’ve outlined above. I can’t let those many hours of screwing around go to waste! Prospective employers and/or clients love to hire people who do Science, not screwing around. And I think this site will fix that.

My new site is designed is designed so much as to be a portfolio and as a blog. My portfolio is small as I have little to show and my blog has a meager amount of posts. As a foundation though, the site is solid and is totally ready to showcase more of my skills.

Because of Hope

I started the project to fix up my personal site in September 2012.

At that time I had heard about this static site generator called nanoc. I had taken down previous versions of this site since I could not be bothered to maintain the PHP beast that is called Wordpress. Also, there was little hope of me hacking on Wordpress to change its default style to something that better demonstrates my skills. Stock Wordpress themes just do not exhibit confidence in web development. Also, learning PHP is just something I’me not too interested in doing. The horror stories of PHP may be FUD, but I’ve experienced and witnessed them myself in a past lifetime. nanoc answers both of these problems. I can hack Ruby and static HTML pages on a server do not have moving parts. nanoc is written in Ruby and generates static HTML pages for uploading onto a server. Compared to a beast like Wordpress, it is also very transparent. The nanoc guys can better explain their kind of product (static site generators) here. I did have a problem in that I had absolutely no experience with nanoc.

In September 2012, I answered a request for help from an organization named Because of Hope at the beginning of my last quarter at UCSB. Because of Hope is an organization that helps children and women in Uganda through sustainable methods. Here’s the email that was sent to the UCSB Computer Science jobs list:

My name is Natalie Lemonnier and I graduated from UCSB in 2011. Since graduating, two of my friends and I have started a nonprofit here in Goleta called Because of Hope (BOH) that is working to empower women in Uganda. I am emailing you because our nonprofit is getting ready to launch a new Student Sponsorship campaign, which will require making some changes to our website, and I was wondering if you knew of anyone (student, faculty or otherwise) who might be interested in helping us make these changes. I am not sure if websites are your area of focus, but I just wanted to contact you and see if you could point us in the direction of someone who might be able to help.

Originally, our friend built our site from scratch. However, he is no long working with us and none of us have any web background to make the necessary changes. I believe that some of the fine-tuning will be relatively easy, while other issues may be more time intensive.

However, because we don’t have web background, we are not really sure of the exact time needed to complete the desired changes. If you or anyone else you know would be interested in volunteering to help us, or would like to learn more about our situation, we would greatly appreciate it.

Because we have only been operational for a year, no one in our organization currently gets paid, so unfortunately it’s not in the budget to pay someone to help us with our website. But we are hoping to find someone who would be understanding of our situation and wiling to help us further our mission! I know UCSB is getting ready to start up, so please feel free to pass on this information to students you think would be willing and able to get involved. Please let us know if anyone comes to mind! Thank you for your time, and please feel free to contact me via phone if that would be at all more convenient for you! (818) 620-8847. Happy Fall Quarter :)

(If you would like to learn more about our organization or view our site, please feel free at http://www.becauseofhope.org.)

Best of Wishes,

Natalie Lemonnier

So, I needed some experience doing a nanoc site, something nice to put on my résumé, and something to satisfy and fulfill the “help poor children in Uganda” request in the Vim text editor. Because of Hope needed someone to maintain their site. With my résumé as sparse as it is, I called up Natalie and offered to help.

I met up with them and took a look at the setup. They had a $20/month VPS to serve one HTML file. The rest of the “pages” were actually linked tumblr posts. I told them upfront I wanted to test out some tools on their problem before I ran with it on my own personal site. In the meantime, I would maintain their current handcrafted site and check out any technical details such as the domain name, Google Apps, or the VPS itself.

I had nanoc chosen out, but what about the rest? I checked out the then-current website’s code and it had some custom 980px grid system. I couldn’t be bothered to try and understand it. I did try to understand it enough to add a donations page that they immediately needed for their upcoming fundraiser.

On the bright side, all the mockups that Nikki Day of BOH did were in Photoshop. They were far more ambitious than what the current site’s current implementation. The custom 980px grid system was not up the the task. In retrospect, it would have been thrown out anyway. I shudder at what beast I would have had to slay if the original plans had gone through.

I chose to use Zurb Foundation. Zurb Foundation is a front-end framework that is far more flexible than many other frameworks out there. This was the the kind of Framework that could handle Nikki’s designs. I would also get free functionality such as drop-down menus, pretty buttons, photo carousel, and first-class support for SCSS, an extension of CSS. Zurb Foundation’s use of SCSS was very instrumental in adapting the design of Zurb Foundation for Nikki’s designs. Compared to its competitor Twitter Bootstrap, there is far more control over elements on the page, especially the buttons. Also, I got a mobile site for free! BOH didn’t even ask for it but it came with it!

Ever since from that time, I’ve been working on BOH’s nanoc site. I learned how to better compose layouts, the Rules DSL of nanoc, and the whole nanoc ecosystem. I was also working with it to integrate Zurb Foundation into the project.

Remember that $20/month VPS? It has since been replaced with Amazon S3. I also tossed in Cloudflare in front. BOH can take on a DDOS attack of the most epic proportions. Why would anyone attack a non-profit is beyond me but the overkill just tickles my funny bone. And the cost is free to near pennies! This is something only a static site can do. As a vote of confidence in S3, NPR used S3 for transferring data in its election app. I’m not saying that BOH will become as big as the presedential election, but it can definitely handle that much traffic if need be and certianly more than a $20/month VPS. Honestly, I’m just glad I don’t have to manage a Linux server to achieve this goal. It’s really upload and forget.

Also, in doing nanoc, I now realize how much of a pain in the ass it is to update the original site. There was so much copy-paste! How do people do all these busy work?

I learned a lot from doing the BOH site and I am continuing to work on it in my spare time. It’s source code is versioned in a Git repository so I can experiment on it all I want. It’s also open source in a GitHub repository. If anybody wants to peruse it and reuse parts of it, they are more than welcome to do so. I plan to keep it in sync with anything that is a nanoc best-practice and update it with new pages.

Mindflakes

Okay, so I’ve helped a non-profit get their site stuff in shape and beyond.

What about my personal site?

Ever since I’ve started the BOH site project, I’ve probably made 3 false starts in creating mindflakes.com. I wager I wasn’t confident enough in my nanoc knowledge in the previous starts. I had given up or I did not have enough time to really sit down and really start work on my site. Also, I was working part time at a local company in the area at this time. I came back home so exhausted from looking through piles of Ruby and Python that I really did not want to put my finger to the keyboard.

Maybe the fourth time was the charm? I didn’t start the mindflakes project this time. What I did instead was start an example/bootstrap project for a blog that used Zurb Foundation and nanoc. I put some of the best-practices I knew into there and learned some more best practices from various sources around the web. I’m especially proud that the project is completely tied to Zurb Foundation as its Javascript is sourced from its gem. There were plenty of Bootstrap examples out there but I believe that Zurb Foundation better fits with the nanoc philosophy in letting power come out in the right places when needed.

By distilling the framework from the content of my site, I was able to work away from any pressure in having a nice site to present. Just having a good framework to build upon is much more gratifying.

Not only that, I now have a free reusable framework that I may be able to reuse for client projects that don’t involve my site. I don’t have to extract magic or best practice out of my site. I can just rebase anything off of the framework with the way I like it. I plan to contribute anything I think is relevant while building my own site back to the framework.

This is my first long blog post since ever. I wrote this to test out this blog. As of this post, Mindflakes.com is still very much out-of-the-box even though I just built the box. I will customize it and make it look pretty enough to really showcase my work over the next few sessions. I’m going to have to decide what to do about images and what to do with the flexibility that Zurb Foundation offers me.

As for what to write, I think I have that settled.