Debugging Webpack app’s with WebStorm 2016.1

[Update Sep 2016: I just posted a support request with Webstorm asking for an updated tutorial on this subject. It’s Ticket WEB-23528]

It recently became possible to debug both server-side and client webpack applications with JetBrains’ WebStorm. The instructions came from a ticket WEB-20781, but they’re not in one place, so I figured I would write them out.

My application was built using react-starter-kit 0.5.1 – which is the current build as of mid-March 2016. These steps worked for me with WebStorm 2016.1 on OSX 10.10.5.

Update: WebStorm 2016.1 can automatically create the React Starter Kit 0.5.1 when you choose New Project…

Debugging Server-side Code

  1. Clone the react-starter-kit repo, then npm install as usual.

  2. You’ll need to tell webpack to generate source maps when debugging. Look in webpack.config.js for a line containing devtool (near line 204), and change it to:
    devtool: DEBUG ? 'source-map' : false,

  3. You also have to remove references to “hash” from the output filenames. Look in the same file near line 156 for a line like this, and change it to:
    filename: DEBUG ? '[name].js' : '[name].[hash].js',

  4. Create a new Run/Debug Configuration. Choose Run -> Edit Configuration, then add a “Node.js” item. Give the configuration a name, and if it isn’t already there, enter the path to your Node.js binary.

  5. Enter the path to your react-starter-kit project folder in “Working Directory”.

  6. For the “Javascript file:” enter the server-side code in the “build” directory. For react-starter-kit, it’s “build/server.js”

  7. Save the configuration by clicking OK.

  8. Build all the files by double-clicking “build” in the npm panel, or typing npm build in the command line.

  9. Set some breakpoints in the server-side code.

  10. In WebStorm, choose Debug (Ctl-D) for the configuration you created in 4. above. WebStorm starts debugging your server-side code, and should stop at breakpoints as expected.

Debugging Client-side Code

I haven’t needed to try these steps yet, but I understand this is the procedure:

  1. Follow Steps 1-3 above.

  2. Create a new Run/Debug Configuration. Choose Run -> Edit Configuration, then add a “JavaScript Debug” item. Give the configuration a name.

  3. Enter the URL to your app in the “URL:” field. For react-starter-kit, it’s http://localhost:3000

  4. Save the configuration by clicking OK.

  5. Start the application by clicking start in the npm panel, or typing npm start from the command line.

  6. Set some breakpoints in the client-side code. WebStorm should be debugging your client-side code, and should stop at breakpoints as expected.

Do you have comments/suggestions/improvements to these procedures? Leave a comment…

Hacking with React – A great book

I wanted to learn React, and I have been wandering around the Internet, looking at a zillion “how to get started” tutorials. I’m not sure what made me stop and look hard at Hacking with React, but I’m so glad I did.

Hacking with React provides a great combination:

  • a progression of straightforward, short lessons toward a working example program,
  • an interesting application (not another To-Do app, thank FSM!),
  • a Howto for setting up some of the confusing Node.js tooling (webpack, ESLint),
  • introduction to testing with Jest,
  • advice about best Node/Javascript/coding practices,
  • a decent understanding of how all these tools work together, and of course,
  • a great walkthrough of the concepts behind React.

At the end of the forty-some tasks (which take a while to complete if you actually do the work of understanding each lesson) you have a working React application, and a pretty good understanding of the basics of starting and building a React/Node application

The book is free to read, but you should buy this book to reward this excellent work. For $10, you’ll learn a lot.

Cmder for Windows – Alternative to PuTTY

For some unfathomable (to me) reason, Windows doesn’t seem to ship with a SSH program out of the box. I’ve used the venerable Putty in the past, but it makes you do a lot of work for a simple SSH session.

I recently ran across an open-source replacement called Cmder for Windows. Based on ConEmu, it bills itself as a “portable console emulator for Windows”. I find it a great replacement for PuTTY, Console, and Powershell. Cmder has a huge number of benefits:

  1. Cmder allows you to create a ssh session simply by typing ssh user@host — I’m immediately in love
  2. Cmder is easy on the eyes, with attractive fonts, default Monokai color scheme, and resizable windows
  3. If you get the “full” version, you get bash emulation with a Unix-y environment, including ls, grep, and even git
  4. Cmder will run off a USB stick
  5. … and it’s free (MIT License)

screen shot of Cmder window

Check it out at http://cmder.net

Bufferbloat and the Ski Shop

Bufferbloat is undesirable latency caused by a router buffering too much data. It makes your kids say, “The Internet is slow today, Daddy”. It’s caused by routers and other network equipment buffering (accepting for delivery) more data than can be delivered in a timely way. Bufferbloat causes much of the poor performance and human pain experienced using today’s Internet.

First a story…

Imagine a ski shop with one employee. That employee handles everything: small purchases, renting skis, installing new bindings, making repairs, etc. He also handles customers in first-come, first-served order, and accepts all the jobs, even if there’s already a big backlog. Imagine, too, that he never stops working with a customer until their purchase is complete. He never goes out of order, never pauses a job in the middle, not even to sell a Chapstick.

That’s dumb, you say. No store would do that. Their customers – if they had any left – would get really terrible service, and would never know when they’re going to be served. And you would be right.

The Ski-Shop Router

Unfortunately, a lot of network routers (both home and commercial) work just like that fictitious ski shop. And they give terrible service. These routers have a single first-in, first-out queue for packets. All traffic is queued, regardless of whether it’s a big or small packet, whether there has been a lot of traffic from a particular source, or whether things are getting backed up.

Since the router has no global knowledge of what’s happening, it cannot inform a big sender to slow down, or throttle a particular stream of traffic by discarding some data (in the ski shop analogy, sending away a customer who has another long repair job). The dumb router simply accepts the data, buffers it up, and expects to send it sooner or later. To make matters worse, in networking (but not in ski shops), if delays get long enough, computers can resend the data, thinking that the original packet must have been lost. These retransmissions further increase bloat and delay, because there are now two copies of the same data buffered up, waiting to be sent…

This is the genesis of the name “bufferbloat” – the router’s memory gets bloated with buffers of packets. When the router doggedly determines to send that data, it blocks newer sessions from even starting, and the entire network gets slow.

What’s the solution?

The members of the CeroWrt team have been working for the last three years to solve the problem of bufferbloat. We’ve largely succeeded: the CeroWrt firmware works really well. CeroWrt users no longer see problems with “the internet being slow” even when uploading and downloading files, watching videos, etc. We have pushed those changes into the Linux Kernel, and also into OpenWrt mainline, and now its Barrier Breaker release contains the Smart Queue Management (SQM) fixes.

SQM introduces a new queueing discipline called fq_codel (see http://tools.ietf.org/html/draft-nichols-tsvwg-codel-02 and http://tools.ietf.org/html/draft-hoeiland-joergensen-aqm-fq-codel-00 for details) that can detect which flows (streams of data between two endpoints) are using more than their share of the bottleneck link (usually, the connection to the ISP).

SQM divides the traffic into multiple queues, one per flow, and sends packets from each queue in round-robin order. (The algorithm is somewhat more involved, so read the full description for details.) fq_codel also measures the time that each packet has been queued (its sojourn time). If packets for a flow have been in the queue for “too long”, then fq_codel either marks them for ECN (Explicit Congestion Notification), or discards a certain percentage of them, preventing the flow from using more than its fair share of bandwidth on the bottleneck.

Wait a minute – discarding packets? Doesn’t that make things worse?

It does slow the rate for the affected flow, but that is exactly what should happen. If a sender has sent so many packets that they’re building up in the router’s memory, then the router must offer back pressure for that flow by dropping/marking some of its packets.

In the meantime, all the other flows (in their own queues) have their packets sent promptly, since they’re not building up and their sojourn time stays low. This automatically keeps everything responsive: short packets, and those from low-volume flows automatically get sent first. The big senders, whose packets are dropped/marked, will re-send the data, but at a slower rate, bringing the entire system back into balance.

What about Quality of Service (QoS)? Doesn’t that help?

Yes, it helps a bit. If you configure your router for QoS, the router will use that information to prioritize certain packets. Good QoS settings can help certain traffic types by sending it first, ahead of the bulk traffic that’s buffered up. But there are several problems with QoS:

  • It doesn’t solve the problem of overbuffering. The QoS rules allow certain packets to go to the head of the queue. But the buffers from large flows are still there, and will have to be sent at some point. And those buffers will stand in the way of any newly arrived traffic that hasn’t been prioritized.
  • As a corollary, there’s no throttling of the big senders: they don’t get early feedback that they are using more than their fair share of the capacity. If the queue/delay gets large enough, the sender could even retransmit the data, making the overall situation worse.
  • It’s annoying to configure QoS. You have to understand how to configure the router and manually make the changes. This is something that only a network geek could come to enjoy. It’s also a maintenance hassle: if you hear about a new application, it may not work well until you adjust the QoS rules to take it into account.
  • Finally, QoS doesn’t help for the download direction. It can improve traffic being sent from your local (home) network toward the Internet. But if the equipment at the far end at your DSL or cable provider is bloated (and very often it is), then QoS in your router won’t make things any better.

The fq_codel and other algorithms in CeroWrt handle all this automatically. The only configuration parameters are what kind of link you have (DSL, Cable, etc.) and the speeds of those links. You don’t have to adjust QoS settings or make other adjustments.

Is my network affected by Bufferbloat?

Quite possibly – here’s one symptom: If the network works well when no one else is using it (early morning, or late at night after everyone else is asleep), but gets slow when others are on the net, then you are suffering from Bufferbloat. Another symptom is if your voice, video chat, or gaming degrade when others are using the network.

Here’s a more scientific test. The DSLReports Speed Test http://dslreports.com/speedtest runs a latency measurement during the download and upload speed tests. (Most speed test sites only measure a few pings when the line is idle.) If the latency gets high on this test, then your router is probably bloated. You can also check out the Quick Test for Bufferbloat on the CeroWrt site for additional information.

What can I do about this?

Three years of network research have paid off: the networks work great at our houses. Our algorithms have been adopted and implemented in the Linux kernel, other operating systems, and an increasing set of commercial network equipment. Our changes have been pushed in to the OpenWRT project. We are making the code available at no charge, and are encouraging all vendors to embrace it.

Regrettably, nearly every piece of equipment with a network connection – home router, ISP headend and DSLAM, phone, personal computer, laptop, tablet, even big commercial routers and switches – needs to have some form of SQM installed. We now have the technology, and it’s simple, but it needs to be deployed.

TL;DR – What you can do about Bufferbloat at your home

  • Consider installing the stable OpenWrt Barrier Breaker firmware on your router. The luci-app-sqm and sqm-scripts packages include the enhancements that we’ve tested and then pushed into the OpenWrt mainline source code. Use the Supported Devices page to find your router, then read the SQM/fq_codel HOWTO. If you don’t want to do all that…
  • Call your router vendor’s support line. With the information from the DSLReports Speed Test in hand, you can mention that the ping times get really high when up/downloading files, and that it really hurts your network performance. Ask if they’re working on the problem, and when they’re going to release a firmware update that solves it. Leave a comment here with their response – I’d love to hear.

An earlier draft of this note appeared in the Bloat and Codel mailing lists. See https://lists.bufferbloat.net/pipermail/codel/2014-February/000802.html Latest update 3May2015

New CeroWrt Router scripts

I posted a set of scripts that people can use to test, configure and debug their CeroWrt router installations. CeroWrt router firmware is a test bed for learning about and eliminating bufferbloat.

The scripts are available on Github at https://github.com/richb-hanover/CeroWrtScripts. They include:

  • betterspeedtest.sh – a script that emulates the famous (but limited) speedtest.net. This script is better because it measures ping latency during the download and upload, and tells you how bad (or not) the bloat is in your network/router.

  • netperfrunner.sh – a script that emulates the RRUL test suite. It runs on the CeroWrt router (or your laptop) to measure the latency under heavy load.

  • config-cerowrt.sh and tunnelbroker.sh – two scripts for configuring a CeroWrt router to a consistent state after flashing new firmware.

  • cerostats.sh – a script that collects a number of configuration and operational stats that is useful for debugging.

LibreSSL fork of OpenSSL

A small team of well-known developers from the OpenBSD team is working on a fork of OpenSSL, to be named LibreSSL.

This group is going through the OpenSSL source code base and removing old/ancient distributions, reformatting the code to KNF (Kernel Normal Format), removing dead code, fixing bugs and improving the package documentation.

They’re aided by the freedom to abandon old cruft that will never again be used, but there’s a certain amount of enjoyment to be had in reading snarky commit comments such as:

ASN1_STRING cleanup - realloc has handled NULL since I had a mullet
and parachute pants ...

and

This only works on systems where calloc() does the integer overflow
check, but if your system doesn't do this, you need to ask your vendor
WHY THEY ARE 10 YEARS BEHIND IN BEST PRACTICE? 

and

I'm glad to know that Ultrix CC has a bug optimizing switch() statements
lacking an explicit `case 0:' construct. But Ultrix has been dead for more than
15 years, really. Don't give it any reason to move out of its coffin.

and

12 years ago, old_des.h was used to provide compatibility with libdes.
The man page says "Compatibility des_ functions are provided for a short
while" and indeed even the original commit message says "The compatibility
functions will be removed in some future release, at the latest in
version 1.0." So here we are, a short while later.

Now I've only been an OpenBSD developer for 11 years, one year less than
this header has existed, but in that brief time, I've learned a thing or
two about deleting obsolete code. It doesn't delete itself. And worse,
people will continue using it until you force them onto a better path.

(To read more from the complete check-in history, see OpenBSD CVSweb)

I, for one, am glad to see a dedicated team look at this code base with the freedom to make it better, smaller, and auditable. I wish them the best, and have thrown a few simoleons into their kitty.

Better than rm -rf /.Spotlight-V100

My MacBook Pro (10.9.2) was running slowly. I saw frequent spinning beach balls, systemstats was consuming 100% of the CPU on a regular basis, Apple Mail was grinding away indexing files, etc.

The forums at discussions.apple.com contain all kinds of dicey recommendations about rm -rf /.Spotlight-V100 followed by “Works for me!” and “Didn’t work for me!” I’m not averse to wading in with Terminal, but I’d rather get advice from a trusted source.

One of the advantages of having AppleCare (you get it for 90 days after you’ve upgraded to a new OS, such as Mavericks) is that you can call 800-275-2273 and speak to a knowledgeable person. They recommended that I do three things:

  • Boot into safe mode, and let the computer perform a thorough disk and permissions check. (See the Apple “What is Safe Boot, Safe Mode?” article for more info.)

  • Force Spotlight to blow away and rebuild its index.

  • Then let Apple Mail re-index.

To do this:

  1. Boot into Safe Boot mode. Restart your computer and hold down Shift until you see the Apple and the progress bar that indicates that the hard drive is being checked/repaired. When the OS starts up (probably many minutes later), you’ll see “Safe Mode” in red text at the upper right corner of the screen. Restart your computer.

  2. To force Spotlight to rebuild its index, simply open System Preferences, click Spotlight, then the Privacy tab. Add your hard drive (hit the “+” and select the hard drive). Then close System Preferences. Re-open System Preferences, and remove the hard drive from the Privacy tab. This is the cue for Spotlight to begin re-indexing. In a moment, you can click on the Spotlight magnifying glass to see that it’s rebuilding the index. (Rebuilding took about three hours on my system.)

  3. Leave Apple Mail closed during all this. (You don’t want to have them both indexing at the same time – it hammers the processor and prolongs the agony.) When Spotlight finishes its indexing, open Apple Mail to allow it to re-index its cached files. You can continue to work in Apple Mail while this happens.

My system has been much more responsive since I did this, and I didn’t have to try any odd suggestions.

Learning about Single Page Apps

I’ve been looking for a way to sort out Angular, Backbone, Bower, Grunt, Node, NPM, Yeoman and all the other buzzwords that get thrown around for writing a Single Page App for the web. There are many Youtube videos around: these videos stand out as excellent resources.

Leave a comment with a link to other resources, and I’ll add them to this list.

And the H@ckfest results…

What a great weekend! What interesting ideas came from the H@ckfest!

My team, the inforMED project, presented an idea for helping first responders (EMTs, ED triage) to get relevant data on their cell phone/tablet when they first encounter a patient. If the patient has one, the EMT would scan the token (RFID, QR code, NFC, etc) and this would immediately bring up a face sheet that give contact information, current medications, allergies, and advance directives to that they could initiate appropriate treatment.

We won an honorable mention for the CIO Track of the entire competition, and also won a $1,000 prize from Intel. The team members were:

  • Rich Brown
  • Adrian Gropper
  • Jake Mooney
  • Susan Katz Sliski
  • Jessica Wang
  • Joyce Zhang
  • Alice Zielinski
  • Patrick Zummo

What a great team!