Migrating Snowpack to Vite (and Docker!)

About eighteen months ago I migrated a small Javascript app to use Snowpack development tooling. (This was mostly for fun, I had already had it working with Webpack.) Snowpack claimed simple dev tooling with nearly instant updates, using the power of ES Modules. It worked pretty well.

About six months ago, the team that developed Snowpack realized that their efforts had paralleled those of the Vite.js tooling. Vite.js also used ES Modules, and provides a mature code base and strong community support. Since the Snowpack team wanted to work on other projects (Astro), they switched their underlying tooling to use Vite.js.

So… I decided to see what it would take to migrate my Snowpack project to use Vite.js. Everything I had read said that it was easy. Here’s a field report from what was required.

A Side Project – Using Docker: Since this Javascript app was a side project, I have also been using it as a learning environment. I had read Jonathan Bergknoff’s Run More Stuff in Docker that made a lot of sense to me. (The magic of using Docker is that once you’ve created the instance, all the tool and dependency versions remain the same. It’s easy then to hand the Dockerfile to a colleague who can build an identical development environment in a few minutes. It also avoids cluttering my daily-driver laptop with multiple versions of Node, npm, Go, Python, rust, and any number of little-used tooling – they’re all encapsulated in the Docker container.)

So I decided to investigate whether using Docker to create the Javascript tooling would make my life better. Googling around led to Andrew Welch’s vitejs-docker-dev project. It builds a Docker instance with full development tooling (Vite.js, pnpm, hot reloading, etc.) that watches the source files in your local directory for updates. You develop the code using your favorite editing tools. Changes are immediately reflected in your browser/test environment. This is very slick. The vitejs-docker-dev repo has good documentation. It describes a lot of background of how the Docker machine gets built and how to use it.

Update: Andrew Welch (who created the vitejs-docker-dev project) sent me a link to his article about using “Docker for all the things.” It’s a good adjunct to the original “Run More Stuff in Docker” post.

Back to the main story – here are the steps I followed to get my Snowpack project on the air with Vite.js:

  1. Create the Docker instance. Clone the vitejs-docker-dev repo. Then run make docker to put the set of development tools into a new Docker instance. This one-time step takes a few minutes. You may see several warnings (as described in a Github issue) but these don’t seem to be important.
  2. Check the default Vite.js app builds on Docker. Run make vite-pnpm run dev in one terminal window, then run make app-pnpm run dev in a second window (waiting between the commands as described in the README). Click the http://localhost:3000 URL to see if the test Vite app starts up. Edit the index.html file to see if the change is reflected in the web browser. (It should be…)
  3. Customize the /app directory for your app. Copy all your app’s files into the app directory. (I renamed the original to app-old and created a new app directory.) This required a bit of jiggering to adjust between Snowpack and Vite.js, such as:
    • My index.html file for Snowpack was in the public directory; for Vite.js, I moved it to the top-level app directory
    • Copy the package.json and other important directories to app directory
    • Adjust the paths for index.html: Snowpack bundles source files from /src into the /dist directory; Vite.js processes the files directly from /src
    • I can’t remember whether I made other tweaks; it certainly wasn’t odious.
  4. Remove references to the Snowpack modules from the package.json file. Optionally, you could take the opportunity to update the versions of dependencies.
  5. Rebuild the Docker instance (#1 above) and restart the development process (#2 above). Click the link to http://localhost:3000 and your app should begin to run. You’ll probably need to make adjustments, but you’ll be substantially on the air.

TL;DR The process of converting to Vite.js wasn’t very hard (at least, not on my small project). It required a little farbling around, but nothing terrible. The jury’s still out on whether vitejs-docker-dev will make my life better – but I think it just might.

Using Apple’s RPM tool

macOS Monterey ships with a tool that measures the responsiveness of your network connection. It saturates the network with traffic for 20 seconds, then measures the rate of short transactions to compute “Responses Per Minute.” Big numbers (above 2000) mean your network remains responsive when the network is heavily loaded. Small numbers (under 800 or so) mean your network isn’t responsive – potentially caused by bufferbloat.

There’s an iOS version described at https://support.apple.com/en-gb/HT212313

Stuart Cheshire and Vidhi Goel talked about the RPM tool at WWDC 2021. Apple also published an Internet-Draft that describes the RPM technique

Here’s a sample run from my Mac. The RPM tool displays my download and upload speeds (nominally 25mbps), and the number of simultaneous flows required to saturate the link (12, in this case). It shows the responsiveness as 1995 round-trips per minute. That’s really good: the average latency – even during heavy load – only increases a bit above the baseline (idle) 21 msec.

% /usr/bin/networkQuality -v
==== SUMMARY ====
Upload capacity: 22.657 Mbps
Download capacity: 23.755 Mbps
Upload flows: 12
Download flows: 12
Responsiveness: High (1995 RPM)
Base RTT: 21
Start: 11/7/21, 7:18:37 AM
End: 11/7/21, 7:18:47 AM
OS Version: Version 12.1 (Build 21C5021h)
%

Here’s a video that shows the tool in operation: https://youtu.be/e9DUTB9okMA

NameD•Tective — mDNS over AppleTalk

[From the Archives of Amusing Technology…] Back in the ’90s, Dave Fisher and I created NameD•tective, a Macintosh control panel that gave any Mac on the Dartmouth network a static DNS name.

It used Name Binding Protocol to let someone create a DNS name based on their name plus their AppleTalk zone. The DNS name had the form: person-name.AppleTalk-zone…dartmouth.edu. The NameD•tective server looked up the NBP name, and returned the computer’s current IP address.

This screen shot shows an example: cd-changer.kiewit.atzone.dartmouth.edu was a server in my office that distributed information to other developers in the Kieiwt building. NameD•tective probably didn’t get broad use at Dartmouth, but it was a neat demonstration project. It led Dave and me to develop the MacDNS software that was shipped as part of Apple’s Internet Connection Kit. Here’s the page from Dartmouth’s website archived by the Wayback Machine: https://web.archive.org/web/19961220043011/http://www.dartmouth.edu/pages/softdev/named.html

Today, computer naming is much simpler. Modern operating systems let a computer specify a mDNS (multicast DNS) name that can be directly looked up to find a host that provides the service.

Comcast modems decrease bufferbloat

Last month, Comcast released a paper Improving Latency with Active Queue Management (AQM) During COVID-19 that shows that their PIE AQM dramatically decreases lag/latency (by a factor of 10X — from 250 msec down to 15-30 msec.) From the paper (page 13):

 

… As explained earlier, for two variants of XB6 cable modem gateway, upstream DOCSIS-PIE AQM was enabled on the CGM4140COM (experiment) variant but was not available on the TG3482G (control) variant during the measurement period

At a high level, when a device had AQM it consistently experienced between 15-30 milliseconds of latency under load. … [The] non-AQM devices experienced in many cases 250 milliseconds or higher latency under load.

See if you can get an XB6 / CGM4140COM cable modem

WireGuard Vanity Keys

A WireGuard VPN provides a fast, secure tunnel between endpoints. It uses public/private key pairs to encrypt the data.

If you have several clients, you have to enter their public keys into your server. Keeping track of those keys gets to be a hassle, since ordinarily, the keys are essentially random numbers.

I found a great project to help this problem: WireGuard Vanity Address. It continually generates WireGuard private/public key pairs, printing keys that contain a desired string in the first 10 characters. For example, I generated this public key for my MacBook Pro (MBP): MBP/DzPRZ05vNZ0XS3P9tlokZPrLy/1lb1Zsm3du4QA= Note the MBP/ at the start – it makes it easy to know that this is my Mac’s key.

To do it, I ran the wireguard-vanity-address program. Here is sample output:


$ ./wireguard-vanity-address MBP/
searching for 'mbp/' in pubkey[0..10], one of every 299593 keys should match
one trial takes 28.7 us, CPU cores available: 2
est yield: 4.3 seconds per key, 232.30e-3 keys/s
hit Ctrl-C to stop
private qMKPNrCMId59XTn5vgDICUh/QzIfhqZdrZ+XQBIJj2w= public zmbP/YEpC8Zl6MacYhcY1lq126tL2UudFjmrwbl2/18=
private HHtPY8IwGBxQ5OTtJY6GcuFpImXtDp9d187zvI0axFo= public qhIiSMbp/extT5irPy4EJfLRPR9jTzQZHlM15Fo/P2E=
private BEnEu1lVdcRI997nj2uPNGsyCZNPhBTCNfgJuYPPJHA= public hZzmBP/8EthWPOFp5wroEGPeJTHGxZ5KENnMiZvniGY=
private 8HRj+YZfSBnYZn38MPE09W2g03JvRJoGbjlDkHQ0Wnk= public mBP/q2dOd+m457PyKTIvI7MDTuXLCneG6MM0ir9rwRc=
...
private dFE8xsDDWNNNY1OjOIlxQiNVbp7Z6tZhXsaOo/5gPH0= public MBP/DzPRZ05vNZ0XS3P9tlokZPrLy/1lb1Zsm3du4QA=
^C
# This last line contains a public key starting with "MBP/"

For more details, read the github page, and also the issue where the author addresses security concerns about decreasing the size of the key space.

Update: I created a Dockerfile to make it even easier to run wireguard-vanity-address. Check out my personal github repo for details.

Jetpack Publicize: Setting the image and text shown on Twitter and Facebook

I use Jetpack Publicize to share my posts on Facebook and Twitter. This makes it easy to create a single WordPress post and have it show up on social media sites. Those services automatically use the Featured Image of the WordPress post, plus an excerpt of the title/text.

There’s another cool feature: Facebook and Twitter will include a photo and text summary for your site whenever someone types your URL into their post on Facebook and Twitter.

But… It can be tricky to figure out how Facebook/Twitter retrieve the image and the text. (I started this quest because Facebook was choosing the wrong image for the thumbnail. I found my first clue by reading WP Beginner’s article How to fix Facebook Incorrect Thumbnails.) This led to the debugging techniques below.

Both Facebook and Twitter use Open Graph metatags to find the desired image, text, title, etc.  Jetpack automatically sets these OpenGraph meta-tags: og:image tag, og:title, and og:description. It also automatically inserts tags for Twitter: twitter:text:title and twitter:description.

To see what those services will display, each has a Debugger/Validator. To use them, go to the page below and type in your URL.

Facebook Validator: https://developers.facebook.com/tools/debug/sharing/
Twitter Card Validator: https://cards-dev.twitter.com/validator

Problem:  Jetpack automatically sets the OpenGraph tags, but sometimes it chooses values that are not useful, and they cannot be modified within Jetpack. (At least, Jetpack support could not tell me how to set them.)

Update: Using Jetpack 7.3.1 and Yoast 11.3, the conflict between these two plugins appears to have gone away. But this note still describes useful debugging techniques for Facebook and Twitter. (31May2019)

Solution – Part 1: The Yoast SEO plugin – I use the free version – also lets you set the Open Graph tags for both Facebook and Twitter. (In addition, I really like how it helps to optimize my text for search engines.) However, Yoast and Jetpack’s Open Graph tags interfere with each other.

Solution – Part 2: This is a little bit yucky, but you can disable Jetpack’s OpenGraph tags with a bit of code in your child theme’s functions.php file, as described in Remove Jetpack’s Open Graph meta tags This is no longer required – see Update above.

TL;DR: (Too Long; Didn’t Read)

 

RandomNeuronsFiring.com – now live!

I have reworked my blog so that the primary domain name is “Random Neurons Firing” (instead of the pedestrian richb-hanover.com). Same content, but a better name.

I’m also adding a new topic to those I’ve previously covered (“Software, Networking, Life”). Over the last two years, I have gone to many planning and zoning conferences to learn more about how to provide attractive housing within communities. I’ll post my notes from those conferences and workshops here. I need to note that these will be my own opinions, and not those of any public boards to which I might belong.


Feel free to share this post on Facebook, LinkedIn, Twitter, or email by clicking one of the icons below. Any opinions expressed here are solely my own, and not those of any public bodies, such as the Lyme Planning Board or the Lyme Community Development Committee, where I am/have been a member. I would be very interested to hear your thoughts – you can reach me at richb.lyme@gmail.com.

SQLite Date and Time Functions – explained

A while back, I was greatly confused by SQLite date and time functions. It took me a while to figure out what was wrong. (It was my error: I hadn’t observed the rule that dates must have this form “YYYY-MM-DD” – four digit year, two-digit month and day.)

Nevertheless, I found that the documentation wasn’t quite clear, so I wrote up these notes as an adjunct to SQLite Datatypes and the SQLite Date and Time Functions pages.


2.2. Date and Time Datatype

SQLite does not have a storage class set aside for storing dates and/or times.
The conventional way to store dates is as a string in a TEXT field.
These fields can be compared directly (as strings) to determine equality or order.

For other date-as-string formats, see Date Strings on the Date And Time Functions page.

For further manipulations on dates and times, the built-in Date And Time Functions of SQLite convert dates and times between TEXT, REAL, or INTEGER values:

  • TEXT as strings (“YYYY-MM-DD HH:MM:SS.SSS” – with leading zero where required, and four-digit year – a so-called “timestring”)
  • REAL as Julian day numbers, the number of days (with fractional part) since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
  • INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.

Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.


Date And Time Functions

The Date and Time Functions page doesn’t really define the the arguments or the return types, so I make them explicit below.

Timestring: The conventional way to store dates is as a timestring – a TEXT field (e.g., “YYYY-MM-DD HH:MM:SS.SSS”). These fields can be compared directly (as strings) to determine equality or order.

To convert to other date representations, SQLite supports five date and time functions. All take a timestring (a subset of IS0 8601 date and time formats, listed below) as an argument. The timestring is followed by zero or more modifiers. The strftime() function also takes a format string as its first argument.

  1. date(timestring, modifier, modifier, …) Returns the date as a string: “YYYY-MM-DD”.
  2. time(timestring, modifier, modifier, …) Returns the time as a string: “HH:MM:SS”.
  3. datetime(timestring, modifier, modifier, …) Returns a string: “YYYY-MM-DD HH:MM:SS”.
  4. julianday(timestring, modifier, modifier, …) Returns the Julian day as an REAL – the number of days (and fractional part) since noon in Greenwich on November 24, 4714 B.C. (Proleptic Gregorian calendar).
  5. strftime(format, timestring, modifier, modifier, …) Returns the date formatted according to the format string specified as the first argument. The format string supports the most common substitutions found in the strftime() function from the standard C library plus two new substitutions, %f and %J.

… see the original SQLite page for modifiers and legal timestring formats …

Examples

This section replicates the examples of the original page, but includes the results and types of the function.

Compute the current date. Returns timestring.

SELECT date('now');  -- Result: 2018-03-07

Compute the last day of the current month. Returns timestring.

SELECT date('now','start of month','+1 month','-1 day'); -- Result: 2018-03-31

Compute the date and time given a unix timestamp 1092941466. Returns timestring.

SELECT datetime(1092941466, 'unixepoch'); -- Result: 2004-08-19 18:51:06

Compute the date and time given a unix timestamp 1092941466, and compensate for your local timezone. Returns timestring.

SELECT datetime(1092941466, 'unixepoch', 'localtime'); -- Result: 2004-08-19 14:51:06

Compute the current unix timestamp. Returns INTEGER.

SELECT strftime('%s','now');  -- Result: 1520444198

Compute the number of days since the signing of the US Declaration of Independence. Returns REAL – days and fractions of a day.

SELECT julianday('now') - julianday('1776-07-04'); -- Result: 88269.7339379285

Compute the number of seconds since a particular moment in 2004: Returns INTEGER.

SELECT strftime('%s','now') - strftime('%s','2004-01-01 02:34:56'); -- Result: 447519729

Compute the date of the first Tuesday in October for the current year. Returns timestring.

SELECT date('now','start of year','+9 months','weekday 2'); -- Result: 2018-10-02

Compute the time since the unix epoch in seconds (like strftime(‘%s’,’now’) except includes fractional part). Returns REAL – days and fractions of a day.

SELECT (julianday('now') - 2440587.5)*86400.0; -- Result: 1520444280.01899

A Practical Tutorial for SQLite Date Functions

The SQLite document doesn’t really show how to use date functions in actual code. Here is an example of inserting and retrieving dates in a table.

Note: It is good practice to store dates as text in a datestring format – YYYY-DD-MM. The “BMW” entry below is inserted as an integer number of seconds, and doesn’t work right when trying to use the julianday() function

bash-3.2$ sqlite3
SQLite version 3.29.0 2019-07-10 17:32:03
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> create table car_table(car_name text, car_date text);
sqlite> insert into car_table values ("Ford", date('now'));
sqlite> insert into car_table values ("Toyota", date('now','7 days'));
sqlite> insert into car_table values ("BMW", strftime('%s','now'));
sqlite> select * from car_table;
Ford|2019-12-15
Toyota|2019-12-22
BMW|1576431883       <-- uh oh, this is in seconds, not a datestring
sqlite> select car_name, julianday(car_date) from car_table;
Ford|2458832.5
Toyota|2458839.5
BMW|                 <-- uh oh, this isn't a julianday, as expected
sqlite>              ^D to exit

Local by Flywheel won’t start because it’s regenerating Docker Machine TLS certificates

I have been using Local by Flywheel and really enjoying it. It does two things:

  1. You can stand up a development version of a WordPress site on your laptop and horse around with it. It’s fast, you can make experiments, and if it blows up, you can simply regenerate in a minute or two.
  2. Using the (paid) Flywheel hosting, you can transfer your local dev server to their public hosting, and you’re on the air.

I have not used this latter facility, but I’m here to tell you that the first part is pretty slick.

But… I went away from Local by Flywheel for a month or so, then came back to start working on a new site. When I wanted to start it up, I saw a succession of messages stating that it was “Regenerating Machine Certificates” and that “Local detected invalid Docker Machine TLS certificate sand is fixing them now.” This looped apparently forever, and wouldn’t work. Here’s my report on their community forum.

After considerable searching, I found a procedure from one of the developers that seems to do the trick. It involves downloading a new version of the Boot2Docker ISO file, and letting the system re-provision itself. The process involved a) Creating an alias (“local-docker-machine”) for the “Local by Flywheel”s docker-machine binary; b) issuing a series of commands to that alias:

local-docker-machine stop local-by-flywheel
rm -rf ~/.docker/machine/certs
local-docker-machine create local-cert-gen
local-docker-machine start local-by-flywheel
local-docker-machine regenerate-certs -f local-by-flywheel
local-docker-machine rm -f local-cert-gen

These steps caused Local by Flywheel to recognize that the Boot2Docker ISO was out of date. It triggered a download of the new version, and gave the output below. When it completed Local by Flywheel worked as expected. Whew!

bash-3.2$ alias local-docker-machine="/Applications/Local\ by\ Flywheel.app/Contents/Resources/extraResources/virtual-machine/vendor/docker/osx/docker-machine"
bash-3.2$
bash-3.2$ local-docker-machine stop local-by-flywheel; rm -rf ~/.docker/machine/certs; local-docker-machine create local-cert-gen; local-docker-machine start local-by-flywheel; local-docker-machine regenerate-certs -f local-by-flywheel; local-docker-machine rm -f local-cert-gen;
Stopping "local-by-flywheel"...
Machine "local-by-flywheel" is already stopped.
Creating CA: /Users/richb/.docker/machine/certs/ca.pem
Creating client certificate: /Users/richb/.docker/machine/certs/cert.pem
Running pre-create checks...
(local-cert-gen) Default Boot2Docker ISO is out-of-date, downloading the latest release...
(local-cert-gen) Latest release for github.com/boot2docker/boot2docker is v18.09.1
(local-cert-gen) Downloading /Users/richb/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.09.1/boot2docker.iso...
(local-cert-gen) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
Creating machine...
(local-cert-gen) Copying /Users/richb/.docker/machine/cache/boot2docker.iso to /Users/richb/.docker/machine/machines/local-cert-gen/boot2docker.iso...
(local-cert-gen) Creating VirtualBox VM...
(local-cert-gen) Creating SSH key...
(local-cert-gen) Starting the VM...
(local-cert-gen) Check network to re-create if needed...
(local-cert-gen) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: /Applications/Local by Flywheel.app/Contents/Resources/extraResources/virtual-machine/vendor/docker/osx/docker-machine env local-cert-gen
Starting "local-by-flywheel"...
(local-by-flywheel) Check network to re-create if needed...
(local-by-flywheel) Waiting for an IP...
Machine "local-by-flywheel" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
Regenerating TLS certificates
Waiting for SSH to be available...
Detecting the provisioner...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
About to remove local-cert-gen
WARNING: This action will delete both local reference and remote instance.
Successfully removed local-cert-gen

Linking Reservation Nexus and TripAdvisor

Connecting ResNexus and TripAdvisor

We wanted our room availability to show up in TripAdvisor and other online services. There are two basic steps, where you tell Reservation Nexus and TripAdvisor how to find each other’s information:

  • Use Reservation Nexus Availability Exchange to share your room availability
  • Use TripAdvisor TripConnect to link up your business to the Reservation Nexus listings

Note: The business name, postal address, URL, and email must be exactly the same in both ResNexus and TriPAdvisor. Check them before starting this procedure:

On the Reservation Nexus site:

  1. In the ResNexus Settings choose Availability Exchange, near the bottom of the settings (first image below).
  2. In the Availability Exchange page:
    • Click the REGISTER button to register your rooms
    • Click Only share my availability… and check off the desired services. (second image)
  3. Click SAVE. The resulting page (third image below) shows:
    • Your Availability Exchange ID next to the UNREGISTER button
    • The Last full synch time

On your TripAdvisor site:

  1. Log into TripAdvisor
  2. Go to https://www.tripadvisor.com/CostPerClick and click Check your Eligibility. It will show a page naming your property to link to the Cost per Click program. (first image below)
  3. Click Get Connected. You will see a page listing the choices. (second image)
  4. Find “Reservation Nexus” and click it to select, it, then click Confirm. (third image)
  5. The confirmation page (fourth image below) should show property prices for a specific night. This confirms that the connection has been established. Continue with the cost-per-click process with TripConnect.
  6. If you see an error (fifth image), ensure that your contact information for Reservation Nexus and TripAdvisor are exactly the same.

Troubleshooting

  • When it works, the connection between Reservation Nexus and TripAdvisor should happen almost immediately, and you should see the confirmation page listing your property prices.
  • If you had to modify your ResNexus info, then you may need to contact ResNexus to have them re-publish your TripConnect info.
  • Contact Reservation Nexus if the connection has not completed within an hour.