London Underground Map Dashboard

Background

I work on large projects to re-signal parts of the tube.  Our system automates most of the operation, including automated routing and driving of the trains.  Each project re-signals another portion of the tube; either a single line or multiple related lines.  We need to keep a close eye on the systems for failures so we can assist and/or analyse the root cause. I needed something to assist in watching for failures.

During the workday, I am constantly referring to the tube map and am often checking on the status of the lines to keep an eye out for any problems.  This can be done online, but that is a polling approach i.e. I check on the state once in a while.  I wanted a large visible dashboard to clearly show the state of the lines - something that my attention would be drawn to when something changes (interrupt driven). This resulted in tube map dashboard to provide me with real(ish) time data about how the systems are behaving.

A video of the dashboard is below. 

The dashboard gives me real time updates to the published state of each London Underground line.  Changes are immediately visible as the LEDs associated with a line with problems start flashing.  The flashing draws my attention to it.  The dashboard also provides Text To Speech to explain the details behind a problem  

Design

The dashboard is shown below (banana for scale).  The finished product is a useful map for me, even when not powered.  Once powered it shows the status of each line by different animations. Neopixels (hacked cheap fairy light strings from Amazon) representing each station are fully lit for a given line when "Good Service" is indicated by the Transport for London site.  The pixels flash when there is disruption on a given line, e.g. the pixels of the line slow flash between 50% to 100% is severe delays, fast flashing is minor delays, etc.  There is also a small e-paper display on the right hand side which shows the date/time as well as any disruption notices. 

The map is mounted onto a 5mm plywood sheet with 272 holes drilled for all the stations and 11 for the list of lines along the side.  It is designed to hang on the wall, so I built a wooden frame for it which also allowed some space for the Pi Zero W2 and associated electronics.  The neopixels are hot glued into holes drilled into the wood.  They shine through the paper, which works as an excellent diffusion material.  The back of the map is shown below.

The architecture is shown below.  A Raspberry Pi Zero W2 queries the TfL Open Data site to get the status of the lines and then creates animations on the 4 pixel strings (100 pixels each)  as needed via a demultiplexer.  The demultiplexer is needed to drive the 4 strings off of a single pin (pin 18).  This was necessary as they are fixed address neopixels.  

The Pi also makes service announcements if there are any major issues to announce.  This is done via the text to speech engine at Google Translate and a Bluetooth speaker.  

Conclusion

This project took a long time to complete due to the amount of physical work required and I kept having to buy new parts, materials, and tools to complete it.  I am satisfied with the result and it has allowed me to keep better track of problems on the tube lines that I am helping to deliver.  I am tempted to build it again as I am sure I could improve the quality significantly. A desktop size version would be useful for the office as well, maybe only showing the line portion in the bottom right.

Obviously there are an enormous number of possibilities for similar dashboards for different purposes, so this post may be useful for those wanting to display completely different types of data.  

A YouTube video of my presentation to Raspberry Pint (Twitter @RaspberryPint, Facebook Group: Raspberry Pint, Meetup: Raspberry Pint) is available at this link:

https://www.youtube.com/watch?v=vzorWEc36h4

Resources and Parts

This link has information on the use of neopixels. 

https://learn.adafruit.com/adafruit-neopixel-uberguide

I used these fairy lights, which are fixed address neopixels.  They work as per any neopixels, except you can't join multiple strings together like normal neopixels.  The additional neopixels don't operate separately to the other string(s) as they share the address space of pixels 0-99 (commanding pixel 5 lights the same pixel in all joined strings).   

I used Molex SL connectors for the wire to wire connectors.  

and RS-Pro connectors for the board to wire connectors. 

The Demultiplexer is the Toshiba 4051BP. 

TfL Open Data is a very rich source of data available to everyone.  

https://tfl.gov.uk/info-for/open-data-users/

The display is a 2.13 inch Waveshare 3 colour e-paper.  It is very slow, but works fine for occasional updates such as the clock, which is updated once a minute.

  

   

Ongoing Reliability Issues - Water Softener Minder - Pi Zero W

The ongoing story - the little Water Softener Minder that couldn't (at least not for proper lengths of time) ...  

Given the difficulties I was having as per the below Water Softener Minder project, I decided to upgrade the project to Buster with the hope it would improve the reliability. 

It improved the reliability, but not in the way I was expecting.  Buster seems better at dealing with exceptions.  I had decided I would reboot the Pi Zero W if it lost the ability to send out a Telegram message.  Under Stretch, the reboot didn't work as the OS became too degraded due to the problem.  Buster seems better at detecting problems with the system and fails more often.  On the other hand, it seems to fail more gracefully, so the reboot command actually worked.  The net effect is better - when there is a problem, it reboots and sorts itself out.  It does mean a lot more rebooting than should actually happen though.  

After all of that, I realised the problem may be with the Telepot library (version 12.7).  I was using the built in mechanism provided to handle incoming messages.  You define a handling function and tell it to use the function in a thread.  I just had no idea what it was actually doing, particularly with the frequency of checking for new messages for my bot and any retries it would do if things went wrong.  I decided it was better to replicate that function in my own code - I could then better deal with any issues that arose.  

Just waiting for the next failure to gather more insight.  Hopefully it won't be soon...

 

Pi Zero W Reliability Issue

For my water softener minder project, I had a regular failure that puzzled me. Every once in a while, the Pi Zero W (running stretch) would lose communication and never regain it. I had initially assumed it was a momentary processor overloading issue. The neopixels would continue to blink away and looking at the status of the service showed that it was working fine (systemctl status <service name>), except for any comms with the network. Even ping didn't work. It didn't matter how long I waited. I am sure I saw this before during previous projects, but most of the time for my silly projects, who cares? Just restart and move on.

In this particular case, the syslogs show the problem starting around 11:49AM on 16 March - screenshots below. As you can see it starts going wrong at the application level and then complains about 120s timeouts. The result is all IP comms is brought to a halt. It seemed like a bit of an overreaction.

My water softener minder is pretty low CPU usage as can be seen via TOP. Eventually I remembered that I had enabled VNC Connect, just because it always amazes me that I can log into a full graphical interface on something as tiny and wimpy as a Pi Zero W. Disabling VNC Connect seems to have resolved the issue and it has been stable since then.

I would be interested in what people think about this. There is a link to a page that was helpful. I think I will work on reducing I/O, but is there anything else that I should do? I see that you could set the timeout to 0, but is that advisable? Or set it to something much larger? Or just live with what I have as it seems to work?

https://www.blackmoreops.com/2014/09/22/linux-kernel-panic-issue-fix-hung_task_timeout_secs-blocked-120-seconds-problem/

Raspberry Pi Water Softener Minder

Water Softeners are great - we live in London where the water is medium hardness, but even that has a significant impact on the plumbing.  We were constantly scrubbing the scale off the bathtub, toilets, and the shower screen.  It also clogs up pipes and valves.  Hard water also has an impact on how soap foams and how it affects your skin.  We decided to get a water softener and are happy we did.

Water softeners work by passing the water supply through a canister containing resin beads.  The resin beads are conditioned (typically overnight) by sending a brine through the beads.  The brine is discarded through a drain.  For more information, see https://www.water2buy.com/, we ended up with W2B200 model.  

The install was a bit of a pain, but a good plumber put it in within a day.  It certainly would not fit under the sink, so it is in the neighbouring cupboard.  Photo of it unmodified below, which has my reflection in the cover.  It isn't exactly an M. C. Escher https://en.wikipedia.org/wiki/M._C._Escher.  

You may have noticed the two marks written on the water softener.  These indicate the minimum and maximum salt hopper fill levels.  This represents a *lot* of salt.  Roughly 25kg per month - you can see the remnants of the first bag to the right bottom in the photo. 

We have had it for less than a month, so we are keeping a close eye on the salt hopper, but clearly our vigilance won't last.  I decided I wanted some way of keep track of the salt in the hopper.

Project Overview

The project diagram is shown below. A RPi Zero W talks I2C to a Time of Flight Sensor, which provides the distance to the salt.  The Pi calculates how much salt is remaining and displays that on a neopixel strip.  At given hours, it sends a message to Telegram, which forwards that to my phone.  At the moment, it sends messages at 07:00, 18:00, and 20:00.  This is the time period when I will likely be awake and be able to deal with low salt.  It is probably more than needed, but I am still working out bugs, so the messages are a good remote heartbeat.     

A photo and video of the project is included below.  The neopixel effects are only reacting to the measurements.  They use a simple animation as I didn't want anything to flashy, but also wanted to know immediately that everything was still actually functioning. As we use this cupboard almost every day, the quick visual check is useful as a watchdog as well as informative.     

Screenshot of Telegram shown here - note that I can send a small number of commands to get information from the Pi. 

All in all, I am pretty pleased with how it has gone.  It still needs work, but most of the work has been completed.  

Details of the build

Time of Flight Sensor 

I decided to use a Time of Flight (ToF) sensor that I had lying around to measure the distance from the top of the hopper to the salt.  I am using a chinese breakout board version of VL53L0X (about £4 on Amazon).  Additional info on this type of sensor is here https://learn.adafruit.com/adafruit-vl53l0x-micro-lidar-distance-sensor-breakout/arduino-code. The sensor measures the distance to the salt by calculating it via the length of time it takes to get a reflection back from a laser it shot out (hence Time of Flight).  It works great, except it has huge error with salt for some reason!  I added a plate to the hopper as a target.  The addition of the plate meant a correct measurement of 23cm, instead of the incorrect measurement with only the salt at 31cm (35% error!).  Of course there was a debate about which plate to use - see below, which shows the inside of the hopper.  The green cylinder contains all the resin and does the softening.  The salt is pillow shaped.   

The ToF sensor is simple to use.  Connect via standard I2C and use the Adafruit library to get the measurement in mm.  The measurement is then used to calculate how much salt is left.  

Neopixel Stip 

The neopixel strip is a standard 60 pixel/metre strip of WS2812 neopixels.  I am using 22 of them, so everything is adjusted for that.  The standard RPi WS2812 library works fine.  To get better colour representation, I pass in a gamma correction matrix, which adjusts for visual perception.  I use HSV colour scheme to get the gradual effect shown, otherwise it can be difficult to get the RGB values correct.  

Telegram Interface

Looking into the possibilities, I was thinking of a WhatsApp interface, but it didn't seem as easy as it should be.  Twitter is obviously also possible, but I wanted a messaging App.  It turns out a telegram interface is fairly straightforward and loads of information is available.  Some basic logic was required, but a Telegram interface turned out to be very simple.  Two way communication is straightforward and commands can be responded to.  Some information below.    

https://www.hackster.io/Salmanfarisvp/telegram-bot-with-raspberry-pi-f373da

Physical Build

The system is hidden away and I just wanted something cheap and cheerful to package the project.  I used one of my favorite boxes, a Schneider IP55 boxes (80mm x 80mm x 45mm) that are available from various retailers for about £2.  A Pi Zero W with a homemade hat fits fine.  I bolted it to the lid as it gives better access to the Pi.  The box is velcroed to the side of the cupboard, which is very convenient if something goes wrong with the hardware.  Just rip if off the wall, which results in that satisfying velcro sound.  A 2.5m strip is very cheap (~5).   

I put the ToF sensor into the cheap plastic hopper lid and attached it by some velcro and some cable ties.  I will probably build a wood lid to replace the lid which is thin plastic.      

Code 

Code is available on GitHub, but it is still a work in progress at the moment and has not been cleaned up or productised.  Enough caveats?  

https://github.com/Richard-Kirby/water-softener-minder

Conclusion

This is really a simple project that is quite straightforward and was easy to get the initial version working.  As usual, dealing with defects and refining it takes much more time than expected.  All of the tech used is well documented in many good sites, so it is relatively easy to sort through how to put it all together. 

Pi Fighter - Heavy Bag Workouts with your Favorite Characters

Overview

One of the many challenges of working with Raspberry Pis is deciding what to build.  In addition to being able to be used as a  general purpose Linux computer, Raspberry Pis open a world of physical computing.  I started playing with Pis a few years ago and have since built a few control projects, which were great fun and a good start on developing my skills as a Pi hobbyist. After a few of these simple projects and honing my Python skills, I was looking for a bigger challenge.  I certainly found it in Pi FIghter.  It has provided hours of challenge, fun, conversation and exercise. 

I studied karate for years and I always loved doing some bag work, i.e. hitting a heavy bag as if you are sparring with a partner.  I decided to combine the two - a bit of physical computing with a heavy bag.  A lot of Pi projects end up being a bit more complicated than you think.  This was no exception.

Concept  

The concept is essentially a real world old school fighting video game.  The player chooses an opponent and challenges them to a sparring match.  Each player has a certain number of health points that decrement each time the other player lands an attack.  Instead of clicking a joystick or mouse button, the player hits a heavy bag.  The strength of the hit is measured by an accelerometer.  The Pi translates the acceleration of the heavy bag (measured in g) into the number of health points to decrement from the opponent.  One of the players wins once the their opponent runs out of health points.  

As this is intended to be an aid to getting and remaining fit, a few simple features have been added to encourage players to extend their sparring session and to return for more sessions.  A win in a sparring match results in the player regenerating some of the health points lost in the match.  This lets the player take on another opponent and still have a fighting chance of winning.  The player also gets a percentage (2%) of the opponent's health points added to their overall health points.  So if the player starts the session with 100 health points and they beat two opponents with 100 health points, their base health points increase to 104 health points.  This allows them to take on tougher opponents next time they spar.

In action with Richard Kirby

The User Experience

The user is presented with a simple User Interface (see below) that allows the player to choose who to fight.  I've loaded Pi Fighter with old school Star Wars™ characters.  Once the user selects an opponent, Pi Fighter displays a message on the interface and the LED matrix indicating that the opponent is ready.  The sparring match begins when the player first hits the heavy bag.  This just ensures the player is actually ready - they often need some time to put on heavy bag gloves or otherwise prepare themselves. The server reads through files that contain the opponents attacks, which have strength of attack and the timing.  The files were created by modifying logged attacks to reflect the particular character.  Darth Vader hits much harder than an C3-PO and is a lot harder to defeat. 

In action below - in the back garden as it does take up quite a bit of space. 

Son Joh has a go

Once the fight is underway, the Pi displays the fight status (see below), including the strength of the player's attacks and the imaginary opponent's attacks as well as their respective health points.  The player can clearly see the state of the sparring match via the Pimoroni Unicorn Hat HD™.  When the fight completes, the player is informed of the end result via the LED matrix.  A graph of the fight is displayed on the interface.  This graph displays each attack and how the health points decreased throughout the fight.   

The overall effect is a fun, engaging, high intensity workout.  It isn't quite at the point where it is addictive for the casual user, but I am sure some would enjoy using it regularly and improving their strength and sparring skills as they progress. 

A video of it in use is available here.

Design

The high level architecture is shown below.  The Raspberry Pi's starring role is to connect to the accelerometer (MPU-6050), a LED matrix display, and a server, which runs the sparring match.  The server can be run on another Pi or Windows™ laptop, but works fine running on the same Pi (a 3B+ in this case).  I connect to the Pi via VNC Connect as you certainly don't want to connect a screen to interface to the PI - it vibrates violently as the match progresses.    

Pi FIghter Architecture

The software consists of two medium complexity Python programs.  The diagram below shows how it all fits together.  I rewrote it a while ago as an object oriented program with reasonable results (for me).  I won't bother you a description of the code, other than to say it is multi-threaded and makes use of tkinter, queues, and matplotlib.  As the server code is Python, it could easily run on a laptop.  The code is intended to run a web server, but currently only runs locally. 

As Pi Fighter is subjected to significant vibration, the physical build makes use of Loctite™, lock washers, and lock nuts.  I used a Pimoroni Unicorn HD™ LED matrix as it is very bright and capable - a very simple display is needed as it allows the player to understand the state of the sparring match even if it is vibrating wildly.  The whole build has worked perfectly, despite the numerous shocks and high levels of vibration.

Build

The build is quite simple and a bit prototypy.  No custom PCB or anything at the moment.  The build just consists of the Pi on the bottom, a bit of stripboard in the middle, and the Unicorn Hat HD on the top.   I use USB boot for this build - easy to damage a SD card mechanically.

I decided to use stripboard as a first try as it is so quick and easy to work with.  The stripboard also provides opportunities for adding all sorts of LEDs, buttons, etc. should I want to do so.  At the moment, it only pulls out the I2C pins (SDA and SCL), 3.3V, and GND for the MPU-6050 accelerometer.  I used an extra tall header to connect Pi, stripboard, and LED Matrix together.

To keep it all together despite vibration and shock, I used decent quality M2.5 bolts with Locktite and lock washers.

The base is acrylic and is attached to the belt with zip ties.  The USB power cord is attached to the stripboard to prevent the usb power getting ripped from the Pi - a very real possibility and a late addition. 

Future Work and Expansion

There is loads more things to do, just to deal with the bugs and making the code more robust.  At some point, I would also like to improve the physical build - it needs to be more compact.   I should really design a custom board for it.  

In terms of extending the project, there are a number of things that could be done:

• Function to allow players to offer up their fights so you can fight against them.  All the major pieces for this are already in place - just need a way of offering the already recorded fights to others. 
• Function to track users over long periods of times (weeks, months, or even years) so the players can see themselves improve. 
• Additional game features such as levels, unlocking special challenges, specialist attacks, etc.  
• Haptic feedback - I have already built a prototype of this with a Raspberry Pi Zero W.  The W vibrates some mobile phone vibrators to indicate you have been hit. 
• Other exercises - again I have prototyped this via a W - counts number of skips for you. 
• Real time fighting with other players anywhere in the world.  This would be a server function that arranges and manages sparring sessions between players.  There is an issue of calibrating between types of heavy bags, so that would have to be dealt with some how.  Otherwise people will have just deal with the fact a fight could be unfair.

Conclusion

As a Raspberry Pi project, this has been great fun.  Fun to build, fun to use, and fun to show people.   I recently took it the Milton Keynes Raspberry Jam and it was quite a popular project and presentation.  As there are a number of characters, everyone can choose an appropriate character to spar with.  I certainly need to add more characters as many of the old school characters are not familiar to the latest generation.   

As noted in the Future Work section, this sort of project can provide an ongoing challenge for a Raspberry Pi maker.  It can be extended significantly and is complicated enough to continue to learn as you build.  I am sure I will return to work on it at some point, but on to other things at the moment.

In addition to the Milton Keynes presentation, I presented at RaspberryPint (Twitter and Facebook @raspberrypint).  The presentation was at Skills Matter (@skillsmatter).  Video is available here.

Appendix

A few more photos here, including a screenshot if you use VNC Connect via your phone.  

Version 1 of Pi Fighter (left) Version 2.0 (right) 

Control via Phone (Android VNC Connect)

Years since I did a blog

This is my new blog - I was surprised to hear it is still a thing.