It sounds to me like the people operating these excel processes have really poor IT and quantitative skills in general.
These are basic things they screwed up. There is no excuse for that at that level.
It sounds to me like the people operating these excel processes have really poor IT and quantitative skills in general.
These are basic things they screwed up. There is no excuse for that at that level.
And here’s part 2, which is really about modeling standards (ASOP 56)
This has been a big topic of discussion in EuSprIG the last few days…
A Cambridge NHS trust has admitted two historic data breaches, stemming from the accidental disclosure of patient data in Excel spreadsheets in response to Freedom of Information (FOI) requests.
Cambridge University Hospitals NHS Foundation Trust CEO, Roland Sinker, revealed the news yesterday, explaining that the first incident occurred in 2021 but had only “recently” come to light.
“The first case related to data provided in a FOI request via the What Do They Know website. In responding to the request, we mistakenly shared some personal data which was not immediately visible in the spreadsheet we provided but which could be accessed via a ‘pivot table’,” he explained.
The issue was that they shared it with a pivot table… and, as you know, double click and -boom- you can see the underlying data.
The way this data was leaked is almost identical to the far graver breach at the Police Service of Northern Ireland (PSNI) earlier this year. The police service also accidentally shared sensitive information with What Do They Know in response to an FOI request, with the data hidden by a pivot table.
So maybe some of us can see the very annoying practice of data being shared via PDF in response to FOIA requests - they may have been burned in the past from this kind of crap. Rather than risk having too much embedded in whatever database extract by accident, (and not having to explain that what-you-see-is-the-minimum-of-what-you-get-you-might-be-getting-more), PDF (currently) guarantees there aren’t extra data coming along for the ride.
Another reason why I don’t use Pivot tables when there are perfectly cromulent and documentable methods instead.
Also, why didn’t they simply Paste-Special Values the Pivot Table result to a new spreadsheet? Do they know that even exists??
Netflix just put their 2023H1 watch report in a spreadsheet. woo.
https://about.netflix.com/en/news/what-we-watched-a-netflix-engagement-report
Rounded to the nearest 100K minutes per program, it seems.
Removing the programs lacking a release date, unsurprisingly, the programs that were globally available and most recently released got the most play
Some of these appeared in this thread before.
In August 2023, the Police Service of Northern Ireland apologised for a data leak of “monumental proportions” when a spreadsheet that contained statistics on the number of officers it had and their rank was shared online in response to a freedom of information request.
There was a second overlooked tab on the spreadsheet that contained the personal details of 10,000 serving police officers.
A series of spreadsheet errors disrupted the recruitment of trainee anaesthetists in Wales in late 2021. The Anaesthetic National Recruitment Office (ANRO), the body responsible for their selection and recruitment, told all the candidates for positions in Wales they were “unappointable”, despite some of them achieving the highest interview scores.
The blame fell on the process of consolidating interview data. Spreadsheets from different areas lacked standardisation in formatting, naming conventions and overall structure. To make matters worse, data was manually copied and pasted between various spreadsheets, a time-consuming and error-prone process.
ANRO only discovered the blunder when rejected applicants questioned their dismissal letters. The fact that not a single candidate seemed acceptable for Welsh positions should have been a red flag. No testing or validation was apparently applied to the crucial spreadsheet, a simple step that could have prevented this critical error.
In 2021, Crypto.com, an online provider of cryptocurrency, accidentally transferred US$10.5 million (£8.3 million) instead of US$100 into the account of an Australian customer due to an incorrect number being entered on a spreadsheet.
The clerk who processed the refund for the Australian customer had wrongly entered her bank account number in the refund field in a spreadsheet. It was seven months before the mistake was spotted. The recipient attempted to flee to Malaysia but was stopped at an Australian airport carrying a large amount of cash.
In 2022, Íslandsbanki, a state-owned Icelandic bank, sold a portion of shares that were badly undervalued due to a spreadsheet error. When consolidating assets from different spreadsheets, the spreadsheet data was not “cleaned” and formatted properly. The bank’s shares were subsequently undervalued by as much as £16 million.
Federal watchdog calls out USDA’s ‘flawed’ foreign ag land ownership reporting
USDA needs to do a better job of collecting, tracking and sharing data on foreign ownership of U.S. farmland, according to a new analysis from the Government Accountability Office (GAO).
The 61-page report, commissioned by Congress, called USDA’s process for collecting and reporting information Agricultural Foreign Investment Disclosure Act of 1978 (AFIDA) “flawed,” noting that information is still submitted via paper forms to Farm Service Agency county offices.
Once that data is transmitted to USDA headquarters, staff manually enters it into a spreadsheet — one which the GAO found was riddled with errors. According to USDA staff, they are still using the same spreadsheet that an employee created in 2016.
“AFIDA data entry errors often occur because headquarters staff manually enters information from AFIDA forms into a spreadsheet without sufficient internal controls to prevent or identify these errors,” the report stated.
“GAO’s review of AFIDA data current through calendar year 2021 found errors, such as the largest land holding associated with the People’s Republic of China being counted twice.”
Gulp!
An #error in an #excel #spreadsheet inflates the purchase cost of two Hilux vehicles ten fold, the error priced the vehicles at USD 1.65M instead of the correct figure USD 165K.
Looks like a manual entry error inflating the price – maybe forgetful of units, who knows
In a response to public concerns, Abia State Governor Alex Otti addressed the reported allocation of N1.5 billion for two Hilux vehicles in the state’s 2024 budget. The figure, which stirred controversy, was attributed to a software error in the Excel spreadsheet used for budget preparation. The issue came to light during Governor Otti’s visit to Eziukwu Road Market in Aba, where he was expressing solidarity with traders affected by a recent fire.
Governor Otti stressed the importance of government accountability and urged the public to maintain vigilance in monitoring government actions. He acknowledged the Excel error, which inflated the cost by tenfold. The corrected figure for the two Hilux vehicles is N150 million, with each vehicle costing N75 million. This revelation is a testament to the governor’s commitment to transparency and public accountability.
$92 Million Excel Error in Norway’s Sovereign Wealth Fund
Here’s an “anthropological report” on the culture at the bank, relating what went wrong –
Last year (spring 2022) we had an off-site. One of our workshops was on “Mistakes and how to deal with them”. We wrote post-it notes, classifying them into different categories from harmless to no-goes. One of my post-it notes, I remember it vividly, read: Miscalculation of the Ministry of Finance benchmark. I placed it in the category unforgivable. When I wrote that note, I honestly couldn’t even dare to think about the consequences . . . And less than a year later, I did exactly that. My worst nightmare. It was a manual mistake. My mistake. I used the wrong date, December 1st instead of November 1st which is clearly stated in our mandate. The mistake was not revealed until months later, by the Ministry of Finance. They reported back that the numbers did not add up. I did all the numbers once more, and the cause of the mistake was identified. I immediately reported to Patrick [Global Head] and Dag [Chief]. I openly express that this was my mistake, and mine alone. I felt miserable and was ready to take the consequences — whatever they might be.
This is from page 45 and after – “Dealing with Mistakes”
Additionally:
However, the processes, the procedures, the control on investments and calculations of amongst others the benchmark is the core of Risk’s mandate. That the numbers presented by Risk is correct, every time, is their job – and the core of their pride and identity. When gaps in the procedures are identified, these gaps must be addressed. Fast. By good thinking and meticulous work. To restore the Ministry’s confidence, but also their own. Shame is contagious and it clings. The task at hand was complex and comprehensive, it was crucial that the team worked as a team – not a group of people working in parallel. When the organisation resolves crises and ambiguous occasions by dealing with is collectively, it provides psychological safety and healthy teams. Crisis strengthens cohesion or reveals the lack of it (Danielsen 2018).
NBIM is set up to handle volatile markets, loss of money, and professional mistakes on numbers. Crises create a tightening and narrowing of the normal principles: leaders and the organisation switches from its regular set of principles to its regular emergency set. In this case, the number one order effect was a big loss of money. However, they gained another effect which might prove to be even more valid in the long term: They proved they could work as a team. This benchmark miscalculation stress-tested the Risk department, and they learned. Collectively. “It’s easy to speak about psychological safety when everything is going smoothly. The real test comes when things go wrong. I’m really proud of where we are as a team, and an organisation, right now,” the Global Head stated in the interview. Psychological safety comes with good leadership. I have tried my best to discuss the case with the Chief.
Several times. He would hear none of it. “I just did what I had to do. This was teamwork”. “You could have done it differently”. “No”. “You could have dragged the poor chap with you to the Leader Group and Board and thrown him to the lions”. “Of course not!”, he said with an indignantly look, and continued: “Leadership comes with responsibility”.During the crisis, the leaders kept focus on the cause and the employees, not the effect. When leaders walk the talk, they become trustworthy. This crisis proved that they were not only capable of dealing with hard-core professional issues. In Simon’s opinion, they also made new practices. The soft issues, caring for the people involved, was done in a new way. This was locally seen as a culture change.
I would like to know what changes to the processes were made.
I have seen many problems coming from manual processes, but also automatic processes … the issue I’m having here is that it seems that after whoever put in the numbers and reported the index, there had been nobody who checked the result independently.
It wasn’t an Excel error per se, really. They just looked up a value from a wrong date. That has nothing to do with Excel.
To err is human. To really screw things up you need a computer.
I had a spreadsheet that randomly wouldn’t calculate some cell(s) when an input changed. Discovered this “undocumented feature” of Excel when finalizing the reserves one quarter.
Wrote a macro to find and replace every instance of = and that “fixed” the problem.
I once wrote a macro that specifically traversed the cells and forced them to calculate in the order I chose them to get recalculated.
So yes, that is specifically Excel not working properly if one has to do that.
This is why all my spreadsheets have logic that only flows top - down or left to right.
Not your usual Excel F1 complaint:
It is not an exaggeration to say that up to and including at least the initial work on the 2024 Williams, its car builds were handled using Microsoft Excel, with a list of around 20,000 individual components and parts.
Unsurprisingly, ex-Mercedes man Vowles - someone used to class-leading operations and systems – had a damning verdict for that: “The Excel list was a joke. Impossible to navigate and impossible to update.”
…
“When you start tracking now hundreds of 1000s of components through your organisation moving around, an Excel spreadsheet is useless.“You need to know where each one of those independent components are, how long it will take before it’s complete, how long it will take before it goes to inspection. If there’s been any problems with inspections, whether it has to go back again.
“And once you start putting that level of complexity in which is where modern Formula 1 is, the Excel spreadsheet falls over, and humans fall over. And that’s exactly where we are."
The federal Finance department failed to remove hidden tabs in a master spreadsheet, exposing confidential pricing data, a review has found, as vendors called for reform of the $70 billion tender system.
Commonwealth ombudsman Michael Manthorpe found that human error led to the mistaken release in February of sensitive pricing data for all 410 consulting vendors.
Mr Manthorpe found the Finance department had failed to remove hidden tabs in a spreadsheet before sending it to 240 suppliers in February, a basic error.
“This is just very basic stuff,” said Geoffrey Campbell, a partner at accounting and consulting firm Nexia Australia.
“This just needed a simple mail merge. I have sent thousands of letters to people using this technology. It has been around for 20 years. They’re completely out of their depth.”
I actually have no idea what that is or how it would alleviate this problem.
I remember this being advertised as one of the killer features of WordStar and WordPerfect.
EDIT: I just remembered – for a few years, I did my birthday and Christmas thank-you notes with mail merge. My mother was not amused…until my grandfather pointed out that she should just be happy that I was willingly doing thank-you notes.