Python libraries: requests and multithreading.dummy

Today was a two-fer: I found two interesting techniques for working with Python. I needed to test a web server, and wanted to send concurrent HTTP get requests. (The problem was that the web server was failing sporadically when hit by a browser, even though individual requests for pages always worked.) I found two libraries that allowed me to write the test code in well under an hour.

  • requests library: I’ve tried urllib2, and it’s a royal pain to use. My general reaction to that library is, “All I want to do is send an HTTP query, and check the response, and I have to write all this code?” Kenneth Reitz has refactored the API so that simple things are simple, and (I suspect, although I didn’t test it), complex things are possible. Check it out at: http://requests.readthedocs.org
  • multithreading.dummy library: Because I wanted to send multiple, overlapping HTTP requests, I figured I’d have to get into threading the Python program. Yuck… I found a great article on Medium https://medium.com/building-things-on-the-internet/40e9b2b36148 that showed how to use the (little known) multithreading.dummy API of Python’s standard multithreading module to trivially (as in, four lines of code) multi-thread the requests.

Winning Team, Boston BlueButton Innovation Challenge

I was a member of the team that won the “Most Likely to Succeed” award at the MedStart Hackathon at Tufts Medical School (http://tuftsmedstart.com/) Our four-member team (Bryan Bordeaux, Michelle Qi, Elaine Wu, and I) produced a tool that helps a primary care physician by listing recommended screenings for patients.

This was a great event. It was well organized by the MD/MBA program at Tufts. It pulled together about a hundred people – entrepreneurs, software developers, physicians, designers, mentors – to think about issues of exchanging medical records (securely) so that patients can be more in charge of their care.

There’s more…

Digital Analytics Fundamentals from Google

I just finished Google’s “Digital Analytics Fundamentals” course at https://analyticsacademy.withgoogle.com/course. I did OK, too 🙂

The course talks about using Google Analytics to show the performance of your website. The course was a series of short (4-10 minute) videos, with a set of questions after each to let you think about what they just said. In total, it probably took me a little under 6 hours to work through all the lessons: that’s about what Google predicts.

Even though I have worked with Google Analytics for a while, I learned a bunch. Some of the info was about new features they’ve introduced since I last looked; some was about getting more information from the existing tools – better filtering, better reporting. It also told about other facilities that I had never had reason to use.

The course is over closed out now, but they’ll probably run it again. It’s well worth it.

Upgrading to OS X 10.9 Mavericks – Really long install times

Mac OS X 10.9 came out on Tuesday, and I decided to take the plunge and move my 2.3 GHz MacBook Pro to Mavericks. The price (free) is right and there are big claims for its decreased power demands. You can read John Siracusa’s opus describing all the new features over at the OS X 10.9 Mavericks: The Ars Technica Review, but I wanted to detail some of my surprises/impressions.

First impression: Install time was astonishingly long. I ran the ~5 Gigabyte download overnight. It went fine, and when I came to the computer in the morning, it was ready to install. The window said “44 minutes”, and I said OK – I have about 75 minutes before I have to leave. Here’s the timeline:

  • 6:45am – Start the install process.
  • 7:59am – Still shows 4 minutes. OK. I’ll wait.
  • 8:04am – Shows 1 minute left. I gotta leave for work. At least the battery’s fully charged. I put the computer on the floor of the passenger side and let it finish as I drive…
  • 8:22am – I hear the reboot chime. Whew! The Apple appears, with the spinner
  • 8:25am – Finally, I see an OS window – it says that it needs five more minutes to complete installation.
  • 8:33am – I’m almost to work, installation completes, and shows the “Incompatible software” dialog. (The Flip WMV player isn’t compatible.) I close the lid and carry the computer inside.

So it took almost two hours to complete the “44 minute” task. At least most other things worked smoothly. But see my other posts…

Switching to WordPress 3.7 beta 2

I decided to take the plunge and switch from WordPress 3.6.1 to the 3.7 beta 2 version. There’s a plugin that is supposed to make this easy – WordPress Beta Tester – but the current 0.97 version of this doesn’t seem to work.

Fortunately, I found a posting on the support forum that suggests that you can use the 0.96 version. It’s at: http://wordpress.org/support/topic/not-updating-from-361-to-37-41 I did these steps:

  • Back up my database (I’ve been using the free UpdraftPlus)
  • Download WordPress Beta Tester 0.96 to my computer
  • Use the Plugins > Add new and find the link on that page to upload the file
  • Activate the plugin
  • WordPress immediately detects that a new version (the 3.7 beta) is available
  • Let the installation proceed
  • Presto! It’s working

The 3.7 beta’s are supposed to check daily for new versions. I now have WordPress 3.7-beta2-25801 – I’ll find out tomorrow if it worked…

Horsing around with Ghost blogging platform

I had some fun checking out the brand new Ghost blogging platform. Its claims to fame are:

  • It’s a blogging platform that is easy to install and setup
  • It can be customized to look the way you want
  • It uses Markdown for text formatting, with a very sexy real-time Markdown->HTML converter built in. You type Markdown text in the left pane, and the right pane shows a live preview of the posting.
  • It’s free/open source, so you can try it too.

Installation was pretty straightforward. Two comments:

  1. I was confused when I made my first post: after finish entering the text, there’s a “Save Draft” button that has a popup menu that allows you to “Publish Now”. Cool! But it doesn’t actually publish until you click the button a second time.
  2. I also had a problem with running it on my test server. It has dramatically limited RAM (128 MBytes) and it’s also running WordPress, Apache, and a bunch of other stuff. Ghost gave an “weird error 137” message in the console, but I’m fairly convinced that’s just because it’s running out of memory. I wrote it up in  the ghost forums so others can learn from my research. (https://en.ghost.org/forum/installation/1129-npm-err-weird-error-137)

Ghost seems to be a solid initial release, with a dedicated team that’ll drive it forward.

Post another thought from Fargo

I’ve been playing with the Fargo blogging software from Dave Winer at Scripting.com It puts your blog info into an S3 bucket, then has links to WordPress, so that you can use Fargo’s outliner to edit your post, then publish it directly to WordPress.

I just found something else that’s really cool. I moved my “Posts” page to a sub-page (after creating a static home page on my WordPress blog) and the postings automatically followed my “Posts” page. Nice job, Dave!