Format pretty reports in PRQL

I can no longer bring myself to write bare SQL – PRQL makes building queries so easy.

I have become enamored with PRQL – Pipelined Relational Query Language, pronounced “Prequel”. It’s a simple, powerful, pipelined SQL replacement.

From the PRQL Github page: Like SQL, it’s readable, explicit and declarative. Unlike SQL, it forms a logical pipeline of transformations, and supports abstractions such as variables and functions. It can be used with any database that uses SQL, since it compiles to SQL.

I wrote a blog post over on the prql-lang.org site that tells how to make SQL queries “look good” using PRQL. See more »

WireGuard Vanity Keys

A WireGuard VPN provides a fast, secure tunnel between endpoints. It uses public/private key pairs to encrypt the data.

If you have several clients, you have to enter their public keys into your server. Keeping track of those keys gets to be a hassle, since ordinarily, the keys are essentially random numbers.

I found a great project to help this problem: WireGuard Vanity Address. It continually generates WireGuard private/public key pairs, printing keys that contain a desired string in the first 10 characters. For example, I generated this public key for my MacBook Pro (MBP): MBP/DzPRZ05vNZ0XS3P9tlokZPrLy/1lb1Zsm3du4QA= Note the MBP/ at the start – it makes it easy to know that this is my Mac’s key.

To do it, I ran the wireguard-vanity-address program. Here is sample output:


$ ./wireguard-vanity-address MBP/
searching for 'mbp/' in pubkey[0..10], one of every 299593 keys should match
one trial takes 28.7 us, CPU cores available: 2
est yield: 4.3 seconds per key, 232.30e-3 keys/s
hit Ctrl-C to stop
private qMKPNrCMId59XTn5vgDICUh/QzIfhqZdrZ+XQBIJj2w= public zmbP/YEpC8Zl6MacYhcY1lq126tL2UudFjmrwbl2/18=
private HHtPY8IwGBxQ5OTtJY6GcuFpImXtDp9d187zvI0axFo= public qhIiSMbp/extT5irPy4EJfLRPR9jTzQZHlM15Fo/P2E=
private BEnEu1lVdcRI997nj2uPNGsyCZNPhBTCNfgJuYPPJHA= public hZzmBP/8EthWPOFp5wroEGPeJTHGxZ5KENnMiZvniGY=
private 8HRj+YZfSBnYZn38MPE09W2g03JvRJoGbjlDkHQ0Wnk= public mBP/q2dOd+m457PyKTIvI7MDTuXLCneG6MM0ir9rwRc=
...
private dFE8xsDDWNNNY1OjOIlxQiNVbp7Z6tZhXsaOo/5gPH0= public MBP/DzPRZ05vNZ0XS3P9tlokZPrLy/1lb1Zsm3du4QA=
^C
# This last line contains a public key starting with "MBP/"

For more details, read the github page, and also the issue where the author addresses security concerns about decreasing the size of the key space.

Update: I created a Dockerfile to make it even easier to run wireguard-vanity-address. Check out my personal github repo for details.

Jetpack Publicize: Setting the image and text shown on Twitter and Facebook

I use Jetpack Publicize to share my posts on Facebook and Twitter. This makes it easy to create a single WordPress post and have it show up on social media sites. Those services automatically use the Featured Image of the WordPress post, plus an excerpt of the title/text.

There’s another cool feature: Facebook and Twitter will include a photo and text summary for your site whenever someone types your URL into their post on Facebook and Twitter.

But… It can be tricky to figure out how Facebook/Twitter retrieve the image and the text. (I started this quest because Facebook was choosing the wrong image for the thumbnail. I found my first clue by reading WP Beginner’s article How to fix Facebook Incorrect Thumbnails.) This led to the debugging techniques below.

Both Facebook and Twitter use Open Graph metatags to find the desired image, text, title, etc.  Jetpack automatically sets these OpenGraph meta-tags: og:image tag, og:title, and og:description. It also automatically inserts tags for Twitter: twitter:text:title and twitter:description.

To see what those services will display, each has a Debugger/Validator. To use them, go to the page below and type in your URL.

Facebook Validator: https://developers.facebook.com/tools/debug/sharing/
Twitter Card Validator: https://cards-dev.twitter.com/validator

Problem:  Jetpack automatically sets the OpenGraph tags, but sometimes it chooses values that are not useful, and they cannot be modified within Jetpack. (At least, Jetpack support could not tell me how to set them.)

Update: Using Jetpack 7.3.1 and Yoast 11.3, the conflict between these two plugins appears to have gone away. But this note still describes useful debugging techniques for Facebook and Twitter. (31May2019)

Solution – Part 1: The Yoast SEO plugin – I use the free version – also lets you set the Open Graph tags for both Facebook and Twitter. (In addition, I really like how it helps to optimize my text for search engines.) However, Yoast and Jetpack’s Open Graph tags interfere with each other.

Solution – Part 2: This is a little bit yucky, but you can disable Jetpack’s OpenGraph tags with a bit of code in your child theme’s functions.php file, as described in Remove Jetpack’s Open Graph meta tags This is no longer required – see Update above.

TL;DR: (Too Long; Didn’t Read)

 

RandomNeuronsFiring.com – now live!

I have reworked my blog so that the primary domain name is “Random Neurons Firing” (instead of the pedestrian richb-hanover.com). Same content, but a better name.

I’m also adding a new topic to those I’ve previously covered (“Software, Networking, Life”). Over the last two years, I have gone to many planning and zoning conferences to learn more about how to provide attractive housing within communities. I’ll post my notes from those conferences and workshops here. I need to note that these will be my own opinions, and not those of any public boards to which I might belong.


Feel free to share this post on Facebook, LinkedIn, Twitter, or email by clicking one of the icons below. Any opinions expressed here are solely my own, and not those of any public bodies, such as the Lyme Planning Board or the Lyme Community Development Committee, where I am/have been a member. I would be very interested to hear your thoughts – you can reach me at richb.lyme@gmail.com.

SQLite Date and Time Functions – explained

A while back, I was greatly confused by SQLite date and time functions. It took me a while to figure out what was wrong. (It was my error: I hadn’t observed the rule that dates must have this form “YYYY-MM-DD” – four digit year, two-digit month and day.)

Nevertheless, I found that the documentation wasn’t quite clear, so I wrote up these notes as an adjunct to SQLite Datatypes and the SQLite Date and Time Functions pages.


2.2. Date and Time Datatype

SQLite does not have a storage class set aside for storing dates and/or times.
The conventional way to store dates is as a string in a TEXT field.
These fields can be compared directly (as strings) to determine equality or order.

For other date-as-string formats, see Date Strings on the Date And Time Functions page.

For further manipulations on dates and times, the built-in Date And Time Functions of SQLite convert dates and times between TEXT, REAL, or INTEGER values:

  • TEXT as strings (“YYYY-MM-DD HH:MM:SS.SSS” – with leading zero where required, and four-digit year – a so-called “timestring”)
  • REAL as Julian day numbers, the number of days (with fractional part) since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
  • INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.

Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.


Date And Time Functions

The Date and Time Functions page doesn’t really define the the arguments or the return types, so I make them explicit below.

Timestring: The conventional way to store dates is as a timestring – a TEXT field (e.g., “YYYY-MM-DD HH:MM:SS.SSS”). These fields can be compared directly (as strings) to determine equality or order.

To convert to other date representations, SQLite supports five date and time functions. All take a timestring (a subset of IS0 8601 date and time formats, listed below) as an argument. The timestring is followed by zero or more modifiers. The strftime() function also takes a format string as its first argument.

  1. date(timestring, modifier, modifier, …) Returns the date as a string: “YYYY-MM-DD”.
  2. time(timestring, modifier, modifier, …) Returns the time as a string: “HH:MM:SS”.
  3. datetime(timestring, modifier, modifier, …) Returns a string: “YYYY-MM-DD HH:MM:SS”.
  4. julianday(timestring, modifier, modifier, …) Returns the Julian day as an REAL – the number of days (and fractional part) since noon in Greenwich on November 24, 4714 B.C. (Proleptic Gregorian calendar).
  5. strftime(format, timestring, modifier, modifier, …) Returns the date formatted according to the format string specified as the first argument. The format string supports the most common substitutions found in the strftime() function from the standard C library plus two new substitutions, %f and %J.

… see the original SQLite page for modifiers and legal timestring formats …

Examples

This section replicates the examples of the original page, but includes the results and types of the function.

Compute the current date. Returns timestring.

SELECT date('now');  -- Result: 2018-03-07

Compute the last day of the current month. Returns timestring.

SELECT date('now','start of month','+1 month','-1 day'); -- Result: 2018-03-31

Compute the date and time given a unix timestamp 1092941466. Returns timestring.

SELECT datetime(1092941466, 'unixepoch'); -- Result: 2004-08-19 18:51:06

Compute the date and time given a unix timestamp 1092941466, and compensate for your local timezone. Returns timestring.

SELECT datetime(1092941466, 'unixepoch', 'localtime'); -- Result: 2004-08-19 14:51:06

Compute the current unix timestamp. Returns INTEGER.

SELECT strftime('%s','now');  -- Result: 1520444198

Compute the number of days since the signing of the US Declaration of Independence. Returns REAL – days and fractions of a day.

SELECT julianday('now') - julianday('1776-07-04'); -- Result: 88269.7339379285

Compute the number of seconds since a particular moment in 2004: Returns INTEGER.

SELECT strftime('%s','now') - strftime('%s','2004-01-01 02:34:56'); -- Result: 447519729

Compute the date of the first Tuesday in October for the current year. Returns timestring.

SELECT date('now','start of year','+9 months','weekday 2'); -- Result: 2018-10-02

Compute the time since the unix epoch in seconds (like strftime(‘%s’,’now’) except includes fractional part). Returns REAL – days and fractions of a day.

SELECT (julianday('now') - 2440587.5)*86400.0; -- Result: 1520444280.01899

A Practical Tutorial for SQLite Date Functions

The SQLite document doesn’t really show how to use date functions in actual code. Here is an example of inserting and retrieving dates in a table.

Note: It is good practice to store dates as text in a datestring format – YYYY-DD-MM. The “BMW” entry below is inserted as an integer number of seconds, and doesn’t work right when trying to use the julianday() function

bash-3.2$ sqlite3
SQLite version 3.29.0 2019-07-10 17:32:03
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> create table car_table(car_name text, car_date text);
sqlite> insert into car_table values ("Ford", date('now'));
sqlite> insert into car_table values ("Toyota", date('now','7 days'));
sqlite> insert into car_table values ("BMW", strftime('%s','now'));
sqlite> select * from car_table;
Ford|2019-12-15
Toyota|2019-12-22
BMW|1576431883       <-- uh oh, this is in seconds, not a datestring
sqlite> select car_name, julianday(car_date) from car_table;
Ford|2458832.5
Toyota|2458839.5
BMW|                 <-- uh oh, this isn't a julianday, as expected
sqlite>              ^D to exit

Local by Flywheel won’t start because it’s regenerating Docker Machine TLS certificates

I have been using Local by Flywheel and really enjoying it. It does two things:

  1. You can stand up a development version of a WordPress site on your laptop and horse around with it. It’s fast, you can make experiments, and if it blows up, you can simply regenerate in a minute or two.
  2. Using the (paid) Flywheel hosting, you can transfer your local dev server to their public hosting, and you’re on the air.

I have not used this latter facility, but I’m here to tell you that the first part is pretty slick.

But… I went away from Local by Flywheel for a month or so, then came back to start working on a new site. When I wanted to start it up, I saw a succession of messages stating that it was “Regenerating Machine Certificates” and that “Local detected invalid Docker Machine TLS certificate sand is fixing them now.” This looped apparently forever, and wouldn’t work. Here’s my report on their community forum.

After considerable searching, I found a procedure from one of the developers that seems to do the trick. It involves downloading a new version of the Boot2Docker ISO file, and letting the system re-provision itself. The process involved a) Creating an alias (“local-docker-machine”) for the “Local by Flywheel”s docker-machine binary; b) issuing a series of commands to that alias:

local-docker-machine stop local-by-flywheel
rm -rf ~/.docker/machine/certs
local-docker-machine create local-cert-gen
local-docker-machine start local-by-flywheel
local-docker-machine regenerate-certs -f local-by-flywheel
local-docker-machine rm -f local-cert-gen

These steps caused Local by Flywheel to recognize that the Boot2Docker ISO was out of date. It triggered a download of the new version, and gave the output below. When it completed Local by Flywheel worked as expected. Whew!

bash-3.2$ alias local-docker-machine="/Applications/Local\ by\ Flywheel.app/Contents/Resources/extraResources/virtual-machine/vendor/docker/osx/docker-machine"
bash-3.2$
bash-3.2$ local-docker-machine stop local-by-flywheel; rm -rf ~/.docker/machine/certs; local-docker-machine create local-cert-gen; local-docker-machine start local-by-flywheel; local-docker-machine regenerate-certs -f local-by-flywheel; local-docker-machine rm -f local-cert-gen;
Stopping "local-by-flywheel"...
Machine "local-by-flywheel" is already stopped.
Creating CA: /Users/richb/.docker/machine/certs/ca.pem
Creating client certificate: /Users/richb/.docker/machine/certs/cert.pem
Running pre-create checks...
(local-cert-gen) Default Boot2Docker ISO is out-of-date, downloading the latest release...
(local-cert-gen) Latest release for github.com/boot2docker/boot2docker is v18.09.1
(local-cert-gen) Downloading /Users/richb/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.09.1/boot2docker.iso...
(local-cert-gen) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
Creating machine...
(local-cert-gen) Copying /Users/richb/.docker/machine/cache/boot2docker.iso to /Users/richb/.docker/machine/machines/local-cert-gen/boot2docker.iso...
(local-cert-gen) Creating VirtualBox VM...
(local-cert-gen) Creating SSH key...
(local-cert-gen) Starting the VM...
(local-cert-gen) Check network to re-create if needed...
(local-cert-gen) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: /Applications/Local by Flywheel.app/Contents/Resources/extraResources/virtual-machine/vendor/docker/osx/docker-machine env local-cert-gen
Starting "local-by-flywheel"...
(local-by-flywheel) Check network to re-create if needed...
(local-by-flywheel) Waiting for an IP...
Machine "local-by-flywheel" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
Regenerating TLS certificates
Waiting for SSH to be available...
Detecting the provisioner...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
About to remove local-cert-gen
WARNING: This action will delete both local reference and remote instance.
Successfully removed local-cert-gen

Linking Reservation Nexus and TripAdvisor

Connecting ResNexus and TripAdvisor

We wanted our room availability to show up in TripAdvisor and other online services. There are two basic steps, where you tell Reservation Nexus and TripAdvisor how to find each other’s information:

  • Use Reservation Nexus Availability Exchange to share your room availability
  • Use TripAdvisor TripConnect to link up your business to the Reservation Nexus listings

Note: The business name, postal address, URL, and email must be exactly the same in both ResNexus and TriPAdvisor. Check them before starting this procedure:

On the Reservation Nexus site:

  1. In the ResNexus Settings choose Availability Exchange, near the bottom of the settings (first image below).
  2. In the Availability Exchange page:
    • Click the REGISTER button to register your rooms
    • Click Only share my availability… and check off the desired services. (second image)
  3. Click SAVE. The resulting page (third image below) shows:
    • Your Availability Exchange ID next to the UNREGISTER button
    • The Last full synch time

On your TripAdvisor site:

  1. Log into TripAdvisor
  2. Go to https://www.tripadvisor.com/CostPerClick and click Check your Eligibility. It will show a page naming your property to link to the Cost per Click program. (first image below)
  3. Click Get Connected. You will see a page listing the choices. (second image)
  4. Find “Reservation Nexus” and click it to select, it, then click Confirm. (third image)
  5. The confirmation page (fourth image below) should show property prices for a specific night. This confirms that the connection has been established. Continue with the cost-per-click process with TripConnect.
  6. If you see an error (fifth image), ensure that your contact information for Reservation Nexus and TripAdvisor are exactly the same.

Troubleshooting

  • When it works, the connection between Reservation Nexus and TripAdvisor should happen almost immediately, and you should see the confirmation page listing your property prices.
  • If you had to modify your ResNexus info, then you may need to contact ResNexus to have them re-publish your TripConnect info.
  • Contact Reservation Nexus if the connection has not completed within an hour.

WordPress Meetup in Londonderry

I’ll be speaking next month at the WordPress Meetup about the using Docker to host a development WP server on your laptop. Here’s the writeup:

Docker for WordPress

Docker enables developers to easily pack, ship, and run any application (including WordPress) as a lightweight, self-sufficient container which can run virtually anywhere.

For WordPress users, this means it’s easy to set up a lightweight development WP server on your laptop/desktop. You can make and test changes before migrating it to your client’s site. Best of all, if you screw things up, you can simply discard the container’s files and start afresh in a couple minutes. And because it’s running on your local computer, no need to worry about hosting, configuring servers, etc.

Rich will show how to install the Docker application on a laptop, then install and start a WordPress Docker container. The result is the familiar new WP install that you can customize to your heart’s (or client’s) content.

The WordPress Meetup is open to all on Tuesday, 8 May. Sign up at https://www.meetup.com/WordPressDevNH/events/249032144/

How to Write Wiki Pages So People Will Read Them

So you’ve just learned something cool on a new subject, and you want to let the world know about your discovery. You go to the project’s wiki, and jot it all down. But how can you help people read what you’ve written?

When I look at pages on a wiki, I use three criteria to determine whether I want to spend the time to read a page. If I’m convinced that the page has the info I’m seeking, I’ll work hard to understand it. But if I can’t tell whether it’s any good, it’s just faster to post a query to the forum. Here are the questions I ask:

  1. Is this page for me? Does it apply to my situation?

    There are a lot of cues to whether a page “is for me”. Obviously the title/heading of the page is important. But when I’m seeking information, I’m not usually an expert in the subject. I need help to understand the topic, and I look for a description that tells what the page is about. I also look for cues to see if it’s up to date. Finally, I love a page that has an introductory section that talks about the kinds of info that I’ll find on the page.

  2. Does the author know more than I do?

    A number of factors influence this judgement. As you’re aware, there’s a huge range of knowledge level of wiki page authors – from expert to the newcomer who’s excited to document his first discovery. As I scan through a page, I’m looking for facts that confirm what I already know (proving the author has some skill), and then things that I don’t (showing they know more.) Finally, it helps to know that the author is aware of the conventions of the wiki – does it look like other wiki pages? If so, I get some comfort that the author is aware of the way other wiki pages work/look.

  3. Can I figure out what to do?

    My final question about whether a page is useful is whether I can use the information. If it’s a tutorial/howto, I want the steps clearly stated – “step 1, step 2, step 3, then you’re done” If it’s a reference page, is the information organized in a comprehensible fashion? Is it really long? Can I pick out what’s important from incidental info?

The challenge I put to every author is to organize the information in a way that presents the most frequently-sought info first, then figure out what to do with the rest. You might move sections around, or move some information onto its own separate page, coalesce it into an existing/similar wiki pages, or even create forum articles (instead of a wiki page) if the subject is rapidly evolving.

DRY Documentation…

DRY – Don’t Repeat Yourself – is it relevant for documentation? I recently saw this comment on a forum…

I’m not sure how useful it is to remove duplication [from the documentation pages]. It’s not code…

IMHO, duplication in documentation is a couple orders of magnitude worse than duplication in code (and duplication in code is bad) because bad documentation has the power to waste more people’s time.

With code, a single (knowledgeable) developer must take the time to read through the duplicated code to look for subtle differences.

But with documentation, every reader – perhaps hundreds of far less knowledgeable people – must mentally diff the two pages looking for common threads/important items/gotcha’s to try to be sure that they will succeed.

For example, these two documentation pages describe [some procedure…], and each describes an substantially different procedure. I often find the differing explanations so difficult to reconcile that I simply give up (or maybe resolve to come back some day), rather than bricking my router/leaving it inoperable/etc.

So, for common tasks, I believe it is always better to have a single well-curated page that correctly and concisely describes the procedure, instead of having multiple people write their own incomplete, or marginally correct procedure.