🦾 comma three Faux-Touch keyboard

Long arms for those of us with short arms from birth or those who can’t afford arm extension surgery!

Built a faux-touch keyboard for the comma three using a CH552G microcontroller. The keyboard is off the shelf and reprogrammed to emulat a touchscreen.

It’s a bit of a hack but it certainly beats gorilla arm syndrome while driving.

For $5 of hardware, it’s very hard to beat.

The journey started in January 2024 and ended in May 2024. Though it, I had to struggle with:

• CH552G programming
• Linux USB HID Touchscreen protocol
• Instructions and documentation

It’s also been through that period of unintended usage. The Frogpilot project has had users who were surprisingly interested in using it as an alternative for some GM vehicles where there is no ACC button. They use the keyboard to touch the screen where ACC would be.

Part of the work also meant upstreaming to the the CH552G community. While I doubt it’ll have any users, it’s nice to give back to the community. There is now a touchscreen library for the CH552G.

The project is open source and can be found at:

https://github.com/nelsonjchen/c3-faux-touch-keyboard

There’s a nice readme that explains how to build and flash the firmware.

🦾 comma three Faux-Touch keyboard

Long arms for those of us with short arms from birth or those who can’t afford arm extension surgery!

Built a faux-touch keyboard for the comma three using a CH552G microcontroller. The keyboard is off the shelf and reprogrammed to emulat a touchscreen.

It’s a bit of a hack but it certainly beats gorilla arm syndrome while driving.

For $5 of hardware, it’s very hard to beat.

The journey started in January 2024 and ended in May 2024. Though it, I had to struggle with:

• CH552G programming
• Linux USB HID Touchscreen protocol
• Instructions and documentation

It’s also been through that period of unintended usage. The Frogpilot project has had users who were surprisingly interested in using it as an alternative for some GM vehicles where there is no ACC button. They use the keyboard to touch the screen where ACC would be.

Part of the work also meant upstreaming to the the CH552G community. While I doubt it’ll have any users, it’s nice to give back to the community. There is now a touchscreen library for the CH552G.

The project is open source and can be found at:

https://github.com/nelsonjchen/c3-faux-touch-keyboard

There’s a nice readme that explains how to build and flash the firmware.

The replay clipper has been ported to Replicate.com!

https://replicate.com/nelsonjchen/op-replay-clipper

Along with it comes a slew of upgrades and updates:

• GPU accelerated decoding, rendering, and encoding. NVIDIA GPUs are provided to the Replicate environment and greatly speed up the clipper.
• Rapid fast downloading of clips. Instead of relying on replay to handle downloads sequentially in a non-parallel manner, we use the parfive library to download underlying data in parallel.
• Comma Connect URL Input. No need to mentally calculate the starting time and length. Just copy and paste the URL from Comma Connect.
• Video/UI-less mode. Don’t want UI? You can have it.
• 360 mode. Render 360 clips
• Forward Upon Wide. Render clips with the forward clip upon the wide clip
• Richer error messages to help pinpoint issues.
• No more having to manage GitHub Codespaces. Replicate handles all the setup and cleanup for you.

Unfortunately, there is a cost. It’s a very small cost but technically Replicate.com is not free. Expect to drop a cent per render. Thankfully, you have a lot a trial credits to burn through and the clipper can run on a free-ish tier.

End to End Longitudinal Control is currently an “extremely alpha” feature in openpilot that is the gateway to future functionality such as stopping for traffic lights and stop signs.

Problem is, it’s hard to describe its current deficiencies without a video.

So I made a tool to help make it easier to share clips of this functionality with a view into what openpilot is seeing, and thinking.

GitHub repository

It’s a bit heavy in resource use though. I was thinking of making it into a web service but I simply do not have enough time. So I made instructions for others to run it on services like DigitalOcean, where it is cheap.

It is composed of a shell script and a Docker setup that fires up a bunch of processes and then kills them all when done.

Hopefully this leads to more interesting clips being shared, and more feedback on the models that comma.ai can use.

I later also ported this to Replicate here.

Replicate: https://replicate.com/nelsonjchen/op-replay-clipper

https://github.com/nelsonjchen/op-replay-clipper/

📽 Capture short ~30 second route clips with the openpilot UI included, and the route, seconds marker branded into it.

Designed to try to encourage more interesting clips to be shared, especially with newer functionality such as “end to end longitudinal control”.

Originally a bit heavy, so instructions were made for others to run it on services like DigitalOcean/GitHub Codespaces. Since then, the clipper has been ported to Replicate which provides GPUs and a nice WebUI.

The UI rendering is composed of a shell script and a Docker setup that fires up a bunch of processes and then kills them all when done. On Replicate, it is wrapped by a Python script that handles the UI and non-UI processing of video and downloads.