HelloTwo

Overview

Kickstarter promo image, ©Hello

In 2014 a startup called Hello launched their first product on kickstarter the "Sense". This little orb of intricate injection moulded plastic was easily the most fully featured sleep tracking device available to consumers. The business model was solely based around sales with no ongoing service fees. This made the business very unsustainable. After attracting quite a lot of success and breaking into mainstream retail sales Hello folded in 2017. 

Unfortunately, this left the hardware unsupported with no servers to process it's data. The Hello company did initially offer some support for the idea of open sourcing support for their devices but this never got off the ground. Fast forward to 2020 and it has become obvious no one is going to get the original hardware working again. It simply wouldn't make sense in the days of $4 WiFi enabled MCUs to put the effort into reverse engineering the software stack on the Sense. What does make sense is scavenging some of the amazing industrial design and sensors and building a new data logging solution.

The original Sense offers 6 environmental sensors: illumination, humidity, temperature (HTU21D-F), sound, dust (Sharp GP2Y10), and motion, via a bluetooth dongle. I'm only interested in air quality so my work only involves those sensors.

Hardware

Before tearing my device to pieces, I had a look around the internet and stumbled onto Lyndsay Williams' Blog. Lyndsay did a great teardown and inspection of the Sense back in 2015 and his images were a great help to me.

Power board pictured later

To summarize, the Sense is built with 4 PCBs. From bottom up: power conditioning, LEDs, processor, and finally sensor board.

Between the LED board and the processor sits the Sharp GP2Y10 dust sensor.

To avoid self heating, the HTU21D-F is mounted at the bottom of the device near the input air vents. The intent here seems that any heat generated in the device will create an updraft. Any resulting airflow would contribute to keeping the temperature and humidity values reasonably true. Reviewing the layout of the sensors shows that for this project only the power conditioning board is needed. All the others can be removed. This leaves plenty of space to fit a NodeMCU board inside the Sense.

In order to power the replacement MCU and talk to the HTU21D-F, there needs to be several connections to the power conditioning board. Originally, these were made with a flex PCB ribbon cable. Not having access to an appropriate header, the obvious solution was to take inspiration from the insanity that is the Wii Modding community and break out the magnet wire.

Using a multimeter in continuity mode, I probed around the power board and found pads which carried the I2C signals from the HTU21D-F and the USB power signals. The same process could be carried out to interface with other sensors in the device, but as previously mentioned I was only interested in air quality. After soldering magnet wire to each of the pads, I applied a couple of dabs of enamel nail polish to relieve the wires and avoid having to do any later rework.

Wiring to the Sharp GP2Y10 was somewhat simpler. Again the connection is made with a ribbon cable. However, in the case of the Sharp sensor this is a regular wire cable. The MCU end of this cable can be snipped off and soldered to. The Sharp GP2Y10 is a really frustrating device. It's apparently calibrated at factory but relies on two external components (a resistor and capacitor) to control the sensing and readout. This means that the sensor can never be more calibrated than the tolerance of the external components. It would have been extremely easy for Sharp to have integrated these and calibrated the system as a whole. With the magnet wire mess already well underway from interfacing the temperature sensor, I decided to simply float the support components for the Sharp sensor.

With all the sensors wired up, I powered up the board by the USB port on the NodeMCU and tested out my interface code. It turned out that several of the digital pins broken out the NodeMCU have odd reservations and the trigger pin for the Sharp sensor had to be moved. 

After this was complete the sensors worked as expected. The only issue was the really intense LED on the NodeMCU. 

The Sense was designed to include a ring of LEDs and as such, the whole thing lights up blue. That wasn't desirable so the LED was quickly desoldered. After carefully reassembling the device, the hardware work was complete. (I did not bring the USB data lines from the power board up to the NodeMCU so the device must be opened to update the firmware via USB).

Software

For collecting and monitoring the data I looked into a lot of solutions. For this project I wanted long term data storage, low cost, and MQTT as the data transfer mechanism. Adafruit and Particle offer hosted services which meet two of these conditions, but fail to provide the full package. Turning to DIY solutions ELK(Elasticsearch, Logstash, and Kibana) was my first thought. However, while searching around on GitHub I found similar datalogging project by Nilhcem. He uses MQTT, InfluxDB, and Grafana. This is a much better stack and thanks to Nilhcem, you can stand up his logging solution from a Docker Compose file in about 5 minutes. A couple of simple Grafana graph configurations later, and you should have a dashboard like this.

If you have a Hello Sense sitting around and want to replicate what I built, you can pull the Arduino code from my GitHub here: https://github.com/Bostwickenator/HelloTwo.

Troubleshooting

• Pin reservations on the NodeMCU. GPIO 9 and 10 are special and you need to be careful using them http://smarpl.com/content/esp8266-esp-201-module-freeing-gpio9-and-gpio10
• Timing for the Sharp GP2Y10 is very dependant on capacitor value. I suggest graphing the sensor response and picking the highest point of the curve for your individual circuit instead of relaying on the 280 microseconds specified in the datasheet.
• Docker for windows does not boot on startup. solution on StackOverflow
• Accidentally using 3 instead of D3 for a pin operation on the NodeMCU will compile and will hard reset your device each time it is executed.
• Self heating. The NodeMCU produces a surprising amount of heat. With the device in full powered up state the temperature reported was 3 degrees centigrade higher than the true value. To combat this I put the WiFi module into forced power down this reduced the temperature offset to 1 degree which I compensated for in software.
• The Sharp GP2Y10 response values are incredibly noisy. The values bounce around with a standard deviation of at least 3 bits in the 10 bit ADC on the ESP8266. To get around this, I had to collect a lot of samples and apply a Gaussian filter 
• The Sparkfun library for the HTU21D-F crashes the ESP8266.The Adafruit library works perfectly.

Reverse Engineering My Head

A few years ago I got my teeth imaged at the dentist. This resulted in two things. My wisdom teeth getting removed and my possession of a medical image viewing tool from Anatomage. This tool lets you view a single dental cone beam CT file. The tool ships as a single exe file with a spiffy icon and everything.

I was very impressed with the volume visualizations that Anatomage's viewer tool could produce but I wanted more. Not being able to read a file from your own health record with an open source tool didn't strike me as right. So started hacking away at the problem. The first breakthrough is that the Invivo viewer does not memory map part of it's own file directly with the volume data. Instead it extracts that data into your C:\ProgramData\ directory as Patient.inv and then loads it from there. A simple copy paste and we have a version of that file to play with.

Now that we have some data let's figure out what we can about the format. First off I opened Patient.inv up in Visual Studio Code which is generally a bad idea for a 70MB file but this showed me that the file started with XML. In fact the entire BLOB of the volume data is included inside the XML. There are also offset values stored in the XML. Presumably the offset values are used to skip the XML parser over binary data to avoid it throwing a fit.

More good news the XML tells us about the images, their sizes and encodings. However they were all in one large BLOB not one per image. The format specified for the images was J2K. This is a stream of JPEG 2000 data without the normal file container around it. I expected that this would mean that I would have to provide metadata to a JPEG 2000 decoder myself however it turns out that the image size is included in the bytestream itself. The only operation needed after some trial and error is to look for the magic bytes at the beginning of each image and chop the file up on those boundaries. This is not a very robust approach and your mileage may vary. With that said for my file it worked. The resulting JPEG 2000 files could then be fed to a decoder. Reading through one of the image files with a hex editor found that they were produced by the JasPer toolkit. Because of this I decided to choose ImageMagick to decode the images as ImageMagick included JasPer. It turns out that this information is out of date and ImageMagick has swapped to a new library. Luckily this new library handles the JasPer produced files just fine.

I've created a node script to automate the process of extracting the layers into PNG files which you can grab from GitHub here.

With that done we get exciting images like the one below.

A raw PNG file from the stack (exciting right?!)

These images have extremely low contrast but there is data there so now on to creating a 3D model that we can use. At first I was expecting to have to voxalize the data myself. However the Brazilian government came to the rescue with InVesalius 3.1 an amazing tool for processing medical data. We can simply load our PNG files into InVesalius and use it to produce a STL file. From there we can do just about anything we want with the model.

Spooky Christmas ornament anyone?