Confessions of a Data Hoarder: Why Good Data Management Still Matters

Data management system, cloud technology, Internet and business concept.

My name is Donald, and I have a confession to make.

Until yesterday, the data on my home external storage was in a sorry state. Duplicate files, random folders with even more random names… it was chaos. I know some of you are now probably rushing to the bathroom to vomit at my poor data management skills – and for that, I sincerely apologise.

So, is good data management still important in an age of powerful indexing and search tools?

I once read an article suggesting that emails should only be sorted into three simple categories – To Do, Current, and Done – with no further refinement. The idea was that time spent over-organising could be saved by relying on the email client’s search function. To be honest, that does work in some cases.

But in others – especially legal or business-critical scenarios – poor organisation can be disastrous. For eample, the case of Zubulake v. UBS Warburg, a landmark sex discrimination lawsuit. UBS couldn’t locate key emails during the trial, which contributed to them losing the case. The result? Sanctions, legal fees, and a very expensive lesson in the importance of data management.

In my experience, well-organised data often follows a hierarchical folder structure – grouping related files together, making them easier to locate and manage. This benefits both human users and indexing systems. Searching becomes faster, errors are reduced, and data integrity is easier to maintain.

On the other hand, when similar data is scattered haphazardly across random folders, systems, and even storage types – Cat5 cables here, USB drives there, Wi-Fi transfers elsewhere – it can lead to critical issues. Old or incorrect data might be used in analysis. Search indexing slows down. Money and time is wasted

So yes, modern computers are good at finding things – but ultimately, good data management is about helping people. A clear, consistent folder structure allows users to explore, locate, and make sense of their data more intuitively. This allows search tools to become even more powerful.

Displaying the ISS on a Map

I have been doing some Continuous Professional Development, and have been working on my JavaScript and API skills.

Using the MapTiler website, I followed the guide to create a real-time location of the ISS Satellite, and have also managed to include a rocket icon to denote the exact location.

To improve this product, I would like to have an automatic zoom feature, instead of fixed zoom, as the ISS is often over water, with no way to tell where in the world it is currently located.

Additionally and ideally, my next project would be to include locations of multiple objects, such as bus locations, and to include useful data such as location, speed, etc, as a pop-up or legend item.

As always, please let me know your thoughts!

Update: Mark Lanwarne, a former colleague of mine, has tweaked the JavaScript code in order to change the zoom settings when the ISS is over a large body of water. This allows a greater understanding of the ISS location in relation to the nearest countries. I have included his version below.

The Use of AI in the Justice System

During the previous few years, the use of AI has risen exponentially both in the entertainment sphere but also in other mainstream industries and services.

For example, ABBA Voyage has created virtual stars based on real people (known as ABBAtars) in order to present a well-loved show as well as generating large amounts of revenue. While the band members were required for the original motion capture, they did not have to be present at any of the performances in London.

Back in 2020, Robert Kardashian made a speech to commemorate the fortieth birthday of his daughter, Kim Kardashian. Nothing out of the ordinary about that, except for the fact that he had died in 2003, and been resurrected as a holographic animation. He was also very complimentary about his son-in-law, Kanye West, whom he had never met, but who had arranged the reunion through DeepFake holograms.

Now this new technology is having an impact in the legal world. In 2021, Christopher Pelkey was shot dead by Gabriel Paul Horcasitas, while on his way home from a church softball game in Arizona. The perpetrator was later found guilty of manslaughter.

The victim’s sister, Stacey Wales, arranged for the creation of an AI video of Christopher Pelkey, who states that he believes in forgiveness and in a God who forgives. This video was created using real video clips of Christopher during his lifetime, as well as a real photo, which had been taken with an old age filter.

The judge stated that the AI had moved him, and that “his obvious forgiveness of Mr Horcasitas reflects the character I hear about today” and that while the family of the victim had asked for the maximum sentence, the judge noted that “I didn’t hear him ask for the maximum sentence”.

However, the words spoken by the avatar of Pelkey, were not his own. They would have been dictated, written, revised, and agreed by other people, and then fed into the AI model, which would have spoken these words using the image of the deceased victim. While a well educated guess can be made about his future wishes regarding his killer, no one can know for sure what his final statement would have been.

Would the judge have taken into account the avatar’s wishes, if he had requested or demanded the death penalty? Whether this will become more common is hard to guess, but if it does, then it is incumbent on judges passing sentences, to remember that the avatar is not a person, but a well devised personification of certain aspects of their character.

London Santander Bike Locations

For the past month, I have been working on a method for the real-time tracking of the availability of London Santander Bikes. 

I used the website MapTiler for the base mapping and hosting of the GeoJSON file with the bike locations. I also used the Transport for London (TfL) API in order to get the real-time number of standard bikes, e-bikes and the number of empty docks.

Each icon represents the location of a docking station, and uses a colour scheme, with red symbolising 0 facilities available, orange symbolising 1 – 4 facilities available and black for 5 or more facilities available. Within the map window, the colour of the icon is selected by the lowest value of standard bikes, e-bikes and empty docks. Clicking on a bike icon shows the name of the docking station, number of standard bikes, e-bikes and empty docks available.

This method also allows users to search for bikes nearest to their location by clicking on the icon below.

The full website can be found here. As always please let me know your thoughts, and feedback.

Cambridge Real-Time Bus Locations

For the past month, I have been working on a method for the real-time tracking of the buses within Cambridge.

I used the website MapTiler for the base mapping the Open Bus Data website in order to get the real-time location of buses. I also uploaded a GeoJSON file with the bus stop locations, as these would normally be static and not require frequent updates. I had to create a proxy file in order to access the data due to CORS issues.

Each bus icon represents the real-time location of a bus, with the route number in a colour beneath the icon. Each route number has a different colour allowing users to easily differentiate between the routes. I have also used the lollipop icon in order to show the location of the bus stops.

This method also allows users to search for buses nearest to their location by clicking on the icon below.

In the future I will add popups for the bus stops showing their name, and will try to get the colour of the buses to match the colour of the text beneath.

The full website can be found here. As always please let me know your thoughts, and feedback.