Thursday, March 1, 2018

Reverse Engineering My Head

A few years ago I got my teeth imaged at the dentist. This resulted in two things. My wisdom teeth getting removed and my possession of a medical image viewing tool from Anatomage. This tool lets you view a single dental cone beam CT file. The tool ships as a single exe file with a spiffy icon and everything.

I was very impressed with the volume visualizations that Anatomage's viewer tool could produce but I wanted more. Not being able to read a file from your own health record with an open source tool didn't strike me as right. So started hacking away at the problem. The first breakthrough is that the Invivo viewer does not memory map part of it's own file directly with the volume data. Instead it extracts that data into your C:\ProgramData\ directory as Patient.inv and then loads it from there. A simple copy paste and we have a version of that file to play with.

Now that we have some data let's figure out what we can about the format. First off I opened Patient.inv up in Visual Studio Code which is generally a bad idea for a 70MB file but this showed me that the file started with XML. In fact the entire BLOB of the volume data is included inside the XML. There are also offset values stored in the XML. Presumably the offset values are used to skip the XML parser over binary data to avoid it throwing a fit.

More good news the XML tells us about the images, their sizes and encodings. However they were all in one large BLOB not one per image. The format specified for the images was J2K. This is a stream of JPEG 2000 data without the normal file container around it. I expected that this would mean that I would have to provide metadata to a JPEG 2000 decoder myself however it turns out that the image size is included in the bytestream itself. The only operation needed after some trial and error is to look for the magic bytes at the beginning of each image and chop the file up on those boundaries. This is not a very robust approach and your mileage may vary. With that said for my file it worked. The resulting JPEG 2000 files could then be fed to a decoder. Reading through one of the image files with a hex editor found that they were produced by the JasPer toolkit. Because of this I decided to choose ImageMagick to decode the images as ImageMagick included JasPer. It turns out that this information is out of date and ImageMagick has swapped to a new library. Luckily this new library handles the JasPer produced files just fine.

I've created a node script to automate the process of extracting the layers into PNG files which you can grab from GitHub here.

With that done we get exciting images like the one below.

A raw PNG file from the stack (exciting right?!)

These images have extremely low contrast but there is data there so now on to creating a 3D model that we can use. At first I was expecting to have to voxalize the data myself. However the Brazilian government came to the rescue with InVesalius 3.1 an amazing tool for processing medical data. We can simply load our PNG files into InVesalius and use it to produce a STL file. From there we can do just about anything we want with the model.

Spooky Christmas ornament anyone?

Monday, July 3, 2017

Lomo To Gif

Lomo To Gif is a little tool that I made to process Octomat photos into little gif flipbook like animations. It's super simple and super meh code but it's available on GitHub here.

Use it in practice here.

Thursday, May 11, 2017

Sigma 150-600 Sport Comfort Handle

The Sigma 150-600 Sport lens is a beast at 2.8kg. This makes it very hard to handle handheld and indeed most people will use this lens on a monopod or tripod. I'm more of a run and gun photographer though so I took matters into my own hands, literally. Introducing the Sigma 150-600 Sport Comfort Handle. Now built for humans, you know the things with fingers.

This thing is a pretty large part which would be expensive to print completely solid. If you want to do that however you can buy one on Shapeways here.

To make a small production run I decided to use a cheap FDM print with low infill to make a mold. This part would not be strong enough to be functional but does a great job of being a positive of the shape. The mold was constructed out of silicone rubber and then a grip was cast with fiberglass reinforced epoxy. The resulting part is EXTREMELY strong and after a coat of paint looks almost exactly like the renders.

Tuesday, December 20, 2016

Shopping for Frameworks

There is nothing more nerve wracking in a software project than choosing which horse to back, which basket to put your eggs in, and which metaphors to mix. Shopping for frameworks (and it truly is shopping, you are going to spend money) is much like regular shopping. If you get in and out of the store in 10 minutes you've probably not thought though how that jacket is going to match your wardrobe and if you got something cheap you'd better be prepared to stitch the buttons back on. So with no further ado I present my humble advice on the titular topic.

Alex's Framework Wisdom

  1. There is no free lunch. You will pay for free frameworks with ongoing development cost working around their oddities and/or contributing upstream. That is to say less CapEx means more OpEx. In the case of subscription licences it's purely OpEx as salaries or as licenses, either way you'll pay.
  2. The quality of the vendor is as important as the framework. You are piling your developer hours (your money) on top of these frameworks. The vendor has the ability to kill support at any time and transmute your beautiful codebase into a pile of technical debt with the snap of their fingers. If that worries you, get a contract that stipulates they won't.
  3. New isn't automatically better. New languages, new toolchains, new companies. On the surface these may be exciting and they may give you a market edge in some cases. Each one of them is however a double edged sword. These are a lot of companies out there using tried and true technologies because they work and they are known quantities. You should be one of these companies unless there is a pressing market reason that you cannot be. Using new things will make it harder to hire staff. The staff you do hire will create technical debt as they learn what works and what doesn't.
  4. Invest in training. When you choose a framework get your development team together to learn about it. It is crucial that this happens as a group. Received wisdom does not work well in the software industry. If all your team learns about something at once it will create homogeneity and accepted best practices. Handing a rulebook to developers will not work well they'll either ignore it or they'll all make disparate readings of it.

How to avoid choosing poorly

Congratulations you think you've found just the right combination of lego pieces to solve your business case. Not so fast! Before committing you (and by this I mean not you, I mean your staff) need to take it for a test drive.
  • Apply at minimum your own standards. What does the framework's source look like? Is it public? If it's public review it. Makes sure it would pass the standards you set for your own code. If possible try fixing an open bug and getting it accepted upstream. If the framework is closed source you are especially dependent on the vendor. Look at the release notes and review the bug tracker. Make sure things are dealt with in a timely fashion. 
  • Consider the vendor's bus factor (how many of their team could get hit by a bus without impacting their ability to deliver). If it's a small number take this in mind. Will their business model keep them around at least as long as your product?
  • Check out the debugger. Write something with a bug in it and hand it to a second developer. Their job is to document how the debugging tooling works while they find the bug. Did it run right out of the box or did they have to spend an hour setting up an environment? Does it hit every breakpoint? Are values presented in a useful way? Can it attach to a running process? Plus any other metrics your team might find useful here.
  • Eyeball the UI. If it has a UI component take it to your UI and branding team. The default themes will not satisfy your business requirements. Get the UI team to specify the changes that will be required and then attempt to implement them. If this is difficult start worrying.
  • Inspect the toolchain. Does it integrate with your CI setup? Can you make repeatable builds of your code over time. If not what would you have to do to internalize any build dependencies? Does it build in 1 second or 1 hour?
  • Figure out how to test the end product. Does the framework have interfaces to test tools? Can you write a unit test, a functional test, and an integration test? Are the tests stable? If the framework has a UI component can you automate the UI testing or will you be paying people to click buttons?
  • Consider reuse. Is the code you write in the framework coupled to it? Will you be able to take code from it to other places in your business? Will you be able to bring code into it?
These are just some high level acid test questions you should be asking yourself when you are looking at a framework. Please take into consideration any other factors important to your staff and your business case.

Case study; Death by 1000 cuts (or Appcelerator Titanium)

The small issues hurt. It is not the big problems that make you regret your choices. It is the day to day grind. The subtle bug that no one suspected and no one documented. It is the lost hours in every week that your staff could have been doing something that generates revenue. See wisdom #1. Trivial hindrances also create morale issues. Monotonous and frustrating work will demoralize a group of people who typically pride themselves on doing new, interesting, and challenging things. Nothing is less rewarding that realizing you lost your day to a missing semicolon.

Below are replies to some issues I have personally raised against a particularly hodgepodge framework known as Appcelerator Titanium. They are prime examples of the kind of bugs that will decimate your productivity should they occur with regularity. Also note these aren't even bugs with the framework these are bugs with the associated toolchain, remember to investigate the whole package you are buying into.

It's a little complicated. The available Titanium SDK releases depends on if you're using ti sdk vs appc ti sdk. appc ti sdk shows the latest and greatest SDKs. ti sdk shows the latest release before the switch from the Titanium CLI to the Appcelerator CLI.

The list of releases for the Titanium CLI is no longer automatically updated, but rather it's a manual process. Because we haven't determined which past releases should be available to ti sdk, we simply haven't updated the release list.

However, the list of releases that appc ti sdk reference is automatically updated and current, so I advise you just use appc ti sdk for now.

Time lost: 1 hour

Smart quotes are not valid in XML. I wonder what the XML parser library is doing. I wouldn't be surprised if the platform was being parsed as "“iphone”". I don't think we should naively replace all smart quotes with ascii quotes and properly scrubbing the smart quotes is more effort than it's worth. In other words, I don't think we're going to improve this.

Time lost: Half day

Trivial issues like the ones above and dozens more will slow your development process and make your project management unpredictable. This has a real fiscal impact. If a developer has a high chance of running into a framework bug or a hidden issue they will not be able to provide accurate estimates of effort to management. Without accurate estimates you will either miss release dates and stress your employees trying to make back time or have long release cycles and lose agility. Neither of these is going to help you make money.


You never know everything about the ship you are getting on board. With that said a few sanity checks may save you months of rowing back to shore or having to listen to an 8 piece band play you out. Spend the time up front to get an idea of what is going on, trust the people who will be doing the day to day work, and remember nothing is free.

Monday, December 19, 2016

Oven Fresh Nexus 5X

Woe and calamity. Yes, I got the dreaded Nexus 5X bootloop issue a couple of weeks back. Having thrown my phone in the freezer (inside a zip-lock bag) I got a boot out of it. That points to the issue being a bad solder joint. Maybe not in the general case but in at least mine. If your device passes a similar test you can try reflow soldering your device by following along below. Warning: this process will void any warranty you have, only do this as a method of last resort if Google or LG will not service your phone. You may also want to consider taking it to a professional electronics repair shop for reflow work.

With that warning out of the way and the freezer test having roused my suspicions I pulled the case off and took the motherboard out. iFixit has a great guide here for that process.

Time to cook!
With the motherboard out I inspected the teardown photos also provided by iFixit and determined which side of the board the packages (chips/black squares) I was interested in were on. The suspect package is the RAM. A Samsung K3QF3F30BM-QGCF with CPU the Qualcomm Snapdragon 808 conveniently located directly beneath it. The constant flexing of the phone in my pocket has most likely cracked one of the tiny BGA solder connectors off the motherboard underneath one of those packages.

Now that you have your motherboard separated from the phone's chassis preheat your oven to 195 degrees Celsius or 390 Fahrenheit whichever is appropriate for your current locale setting. While your oven is heating take out some aluminum foil and crush it into a ball. Un-crumple it so that there is still a rough texture as shown. This is going to help limit the heat transmission to the underside of the motherboard which we don't want to reflow. Place the motherboard on the foil and press it down. At this point attempt to get the board laying as flat as possible. You don't want parts sliding down the board on an angle if you overcook. Place the foil and motherboard on an oven safe cooking surface, a ceramic casserole dish will do nicely so long as it has a flat bottom, I used the spill tray from a waffle iron. So pick whatever looks good to you. Season with a twist of lemon and cracked pepper.

Now that your oven is up to temperature place the dish into the oven and start a timer for 6 minutes and 30 seconds. Wait anxiously. Note: If your oven is fan forced you may need to make an adjustment to the cooking time.

Remove your motherboard and let it rest until completely cool. Inspect the board for any physical damage, if you've really messed it up be extremely careful with reconnecting the battery. Who knows what you might have shorted out.

Reassemble your phone following the guide from iFixit and give that puppy a charge. Power it up and hope for the best!

Friday, July 15, 2016

Minolta 7000 Battery Upgrade MK2

So my last post was about building a replacement battery pack for the Minolta 7000. Since then I had a much higher quality 3D print made of the housing with some lessons learnt from the first version. This is the result.

This part was ordered from Shapeways and I'm quickly learning that they are so far ahead in this game that you should just go to them straight away. Incidentally you can buy this part from there if you want to build this.

Need some help building that? Well tough luck! I present to you the world's worst build log. Seriously everything that could go wrong with this video went wrong. I don't think I even build a single thing on camera.

So I might not be a videographer but you have to admit the end product looks pretty nice.

Monday, May 23, 2016

Minolta 7000 Battery Upgrade

Say you've got one of these Minolta 7000 cameras. Great camera. Just one problem. Whenever you pick it up the batteries are flat. You've lined up your shot, you press the shutter button, the mirror flips up and nothing happens.  

What you need is something a little more reliable than these original battery grips. The piles of dead AAA and AA batteries get a little annoying after a while. 

The solution as I see it is having a rechargeable battery with a built in charge controller. This means the camera can constantly sit on power while it is at home. Always 100% charged ready to run a few rolls of film. Luckily the engineers at Minolta made this battery pack design very modular and with quite a simple electrical interface. First order of business is to model the battery holder so that we don't have to destroy an original part. 

You can note the that this design doesn't have any ports or holes. This was because it was being produced before I had the rest of the components in hand. Speaking of components these are the parts that you need:
  • TP4056 based charge controller
  • A lithium battery (This one fits easily)
  • A couple of 3mm LEDs why not buy 30!
  • Some brass rod to make your contacts. Springs would be even better but good luck buying two of those for anything less than $10. I cant justify that to myself somehow.
  • And finally some jumper wire for connecting everything up.

Implementation time! It is a really simple setup. Simply solder the battery to the appropriate terminals. Desolder the existing LEDs from the PCB and replace them with the new 3mm ones. I used the jumper wire to make a flexible connection here. Now drill and rasp out the holes in a spot that you like. The top side of the new battery grip makes the most sense because the camera can sit flat while you are charging it and your hands won't come in contact with the ports while you use the camera.
Here is the result of this build out. I've filled a lot of the grip with Sugru. This fixes the components in place extremely solidly. This is particularly important for the charge controller as it has to stand up to the force of repeated USB connection and disconnection cycles. The Sugru was also used to fill up void space from the imperfect fit of the part. If you want to do this simply brush vegetable oil onto the side of the void you want to fill attach the Sugru to the other part and press both sides together until they mate the way you want them to.

One thing to note is the green pin. I originally missed this part. It is used to separate the memory backup battery inside the camera from the circuit when the battery pack is connected. Forgetting this will cause your battery to attempt to charge this little 3V lithium cell which could go badly. Another option is simply to take the backup cell out since you wont need to detach the new battery grip ever again. Luckily the new grip looks much better on the camera than it looks inside.

Of course there were a few lessons learned along the path of this build.

For a start voltage wise the camera will accept 4.1V and nothing lower. Below this point it automatically shuts down when you press the shutter button. This was tested on a beefy bench power supply so it's not the battery that causes this behavior. The camera draws about 1.1 amps during it's shutter actuation and frame winding. This load is for a roughly 200 milliseconds. It then returns to it's quiescent draw which is very low. Oh and before I forget the metering system and the LED backlight for the viewfinder together draw around 300mA when they are on. The AF motor also draws around 1 amp.

On the topic of current you need to make sure that your battery and current protection circuit will supply just over an amp to the camera at peak load without tripping a short protection feature. This makes small AAA size cells with inbuilt protection a bad choice.

Earlier I said the camera only accepts >4.1V with the factory manual suggesting a max voltage of 6V be applied to the camera. So how does it work on a LiPo battery, don't they have a voltage of 3.7V? Well not quite they have a nominal voltage of 3.7V their discharge curve is shown to the right. Luckily for this project it starts at 4.2V. Clearly it drops very quickly to the nominal voltage under discharge. However we are using so little of the battery's capacity this is not an issue in practice.

One thing I learned a particularly hard way about the Minolta 7000 is that they have an internal fuse. The fuse is located inside the hand grip hidden under a film of tape and is formed by a PCB holding a single strand of fuse wire. There is a diode setup to short circuit the battery across this fuse if the polarity is reversed by accident. It's quite a smart setup because in the normal mode of operation you aren't incurring any losses. So if you aren't getting any response from one of these cameras it is a good thing to check before disassembling the entire thing like I did. You can access it by removing a few screws at the bottom of the body and a few inside the front of the grip.