gaaboard.com

Non GAA Discussion => General discussion => Topic started by: RedHand88 on July 19, 2024, 08:47:27 AM

Title: The Great Windows Outage of 2024
Post by: RedHand88 on July 19, 2024, 08:47:27 AM
Anyone else go into no network this morning?
Bit of a handling, looks to be worldwide. Flights grounded, banks affected etc
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 08:50:53 AM
This (https://www.theregister.com/2024/07/19/crowdstrike_falcon_sensor_bsod_incident/) seems a big thing.

All banks in Australia and NZ are down apparently.
Title: Re: The Great Windows Outage of 2024
Post by: Milltown Row2 on July 19, 2024, 09:00:33 AM
Quote from: imtommygunn on July 19, 2024, 08:50:53 AMThis (https://www.theregister.com/2024/07/19/crowdstrike_falcon_sensor_bsod_incident/) seems a big thing.

All banks in Australia and NZ are down apparently.

So far so good with us today.. I certainly don't want to go back when it happened to us, was a complete pain in the hole
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 09:28:32 AM
This is one of the worst internet outages I have seen. I know a few in that company and I would say it's not fun in there this morning  ;D  Dunno how it has been so far reaching - be interested to see the analysis of how it caused such havoc.
Title: Re: The Great Windows Outage of 2024
Post by: Mourne Red on July 19, 2024, 10:05:51 AM
As someone in IT support - It's been hell so far, Crowdstrike/Microsoft going to have big compensation claims against them
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 10:26:48 AM
It's a disaster.
Title: Re: The Great Windows Outage of 2024
Post by: StephenC on July 19, 2024, 10:37:12 AM
Wonder what defines who is affected? We are windows-heavy here but all global systems OK so far.
Title: Re: The Great Windows Outage of 2024
Post by: Taylor on July 19, 2024, 10:48:02 AM
Quote from: StephenC on July 19, 2024, 10:37:12 AMWonder what defines who is affected? We are windows-heavy here but all global systems OK so far.

Very strange how only some companies are impacted indeed
Title: Re: The Great Windows Outage of 2024
Post by: armaghniac on July 19, 2024, 10:57:17 AM
Quote from: StephenC on July 19, 2024, 10:37:12 AMWonder what defines who is affected? We are windows-heavy here but all global systems OK so far.

If you don't use Croudstrike then none of this is relevant to you.

It is a clusterfuk, machines have to be manually rebooted into safe mode to get them going again and that would be a slow process.
Title: Re: The Great Windows Outage of 2024
Post by: tbrick18 on July 19, 2024, 11:11:44 AM
Did I read Windows 10 only affected?
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 11:25:28 AM
That is what I read too.
Title: Re: The Great Windows Outage of 2024
Post by: Brendan on July 19, 2024, 11:46:25 AM
It's the start if the alien invasion
Title: Re: The Great Windows Outage of 2024
Post by: AustinPowers on July 19, 2024, 12:37:00 PM
There was many  a great windows outage during the troubles

Glaziers  couldn't get a minute to themselves
Title: Re: The Great Windows Outage of 2024
Post by: clarshack on July 19, 2024, 12:43:08 PM
Didn't the X-Files predict that this would happen with Banks going offline on a Friday and the chaos that would ensue?
Title: Re: The Great Windows Outage of 2024
Post by: quit yo jibbajabba on July 19, 2024, 12:48:28 PM
Quote from: AustinPowers on July 19, 2024, 12:37:00 PMThere was many  a great windows outage during the troubles

Glaziers  couldn't get a minute to themselves

What have the utd owners got to do with this
Title: Re: The Great Windows Outage of 2024
Post by: naka on July 19, 2024, 01:22:14 PM
access to ticketmaster gaa log in affected
some craic if not fixed by sunday
Title: Re: The Great Windows Outage of 2024
Post by: J70 on July 19, 2024, 01:24:29 PM
Quote from: RedHand88 on July 19, 2024, 08:47:27 AMAnyone else go into no network this morning?
Bit of a handling, looks to be worldwide. Flights grounded, banks affected etc

My work in NYC, pretty much everything relying on a network is down.
Title: Re: The Great Windows Outage of 2024
Post by: Milltown Row2 on July 19, 2024, 01:25:46 PM
Quote from: naka on July 19, 2024, 01:22:14 PMaccess to ticketmaster gaa log in affected
some craic if not fixed by sunday

Seen that.. Though I have the ticket in the wallet (whatever that is)!! Hopefully it doesn't f**k up!
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 01:27:35 PM
Apparently there's a fix out there already and sounds to have been deployed in a lot of places. It'll take a while as this was so far reaching but would expect be grand for then.
Title: Re: The Great Windows Outage of 2024
Post by: RedHand88 on July 19, 2024, 02:17:29 PM
Quote from: naka on July 19, 2024, 01:22:14 PMaccess to ticketmaster gaa log in affected
some craic if not fixed by sunday

How many tickets for a final are through this system?
Club tickets would be paper? If I mind right season tickets were paper for the final but this could be different now.
Title: Re: The Great Windows Outage of 2024
Post by: Saffrongael on July 19, 2024, 02:22:49 PM
I wonder will the scanning system be affected in Croke Park
Title: Re: The Great Windows Outage of 2024
Post by: armaghniac on July 19, 2024, 02:39:25 PM
Quote from: imtommygunn on July 19, 2024, 01:27:35 PMApparently there's a fix out there already and sounds to have been deployed in a lot of places. It'll take a while as this was so far reaching but would expect be grand for then.

"Deployed" means someone going to each machine, as the machines will not start there is no possibility of network deployment. Some places have more people to do the needful than others.
Title: Re: The Great Windows Outage of 2024
Post by: naka on July 19, 2024, 02:40:52 PM
Quote from: RedHand88 on July 19, 2024, 02:17:29 PM
Quote from: naka on July 19, 2024, 01:22:14 PMaccess to ticketmaster gaa log in affected
some craic if not fixed by sunday

How many tickets for a final are through this system?
Club tickets would be paper? If I mind right season tickets were paper for the final but this could be different now.
all premium, season tickets
and a fair amount of club members received codes to download tickets

seems sorted now tbf
Title: Re: The Great Windows Outage of 2024
Post by: AustinPowers on July 19, 2024, 03:03:51 PM
Quote from: quit yo jibbajabba on July 19, 2024, 12:48:28 PM
Quote from: AustinPowers on July 19, 2024, 12:37:00 PMThere was many  a great windows outage during the troubles

Glaziers  couldn't get a minute to themselves

What have the utd owners got to do with this

That's a totally  different   Kind of "troubles"
Title: Re: The Great Windows Outage of 2024
Post by: quit yo jibbajabba on July 19, 2024, 03:38:02 PM
Quote from: AustinPowers on July 19, 2024, 03:03:51 PM
Quote from: quit yo jibbajabba on July 19, 2024, 12:48:28 PM
Quote from: AustinPowers on July 19, 2024, 12:37:00 PMThere was many  a great windows outage during the troubles

Glaziers  couldn't get a minute to themselves

What have the utd owners got to do with this

That's a totally  different   Kind of "troubles"

😉🤓
Title: Re: The Great Windows Outage of 2024
Post by: Orior on July 19, 2024, 07:37:58 PM
Quote from: RedHand88 on July 19, 2024, 08:47:27 AMAnyone else go into no network this morning?
Bit of a handling, looks to be worldwide. Flights grounded, banks affected etc

Great thread title, and affected more people and bugs than the 1666 Great Fire of London.
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 07:40:31 PM
Quote from: armaghniac on July 19, 2024, 02:39:25 PM
Quote from: imtommygunn on July 19, 2024, 01:27:35 PMApparently there's a fix out there already and sounds to have been deployed in a lot of places. It'll take a while as this was so far reaching but would expect be grand for then.

"Deployed" means someone going to each machine, as the machines will not start there is no possibility of network deployment. Some places have more people to do the needful than others.

It'll be slow but it's 36 hours plus and with potential loss of revenue then you would expect the Gaa, ticketmaster etc to be all over it.
Title: Re: The Great Windows Outage of 2024
Post by: RadioGAAGAA on July 19, 2024, 08:26:10 PM
Quote from: Mourne Red on July 19, 2024, 10:05:51 AMAs someone in IT support - It's been hell so far, Crowdstrike/Microsoft going to have big compensation claims against them

Dunno if it has much to do with MS.

Seems like Crowdstrike have either a bunch of cowboys running their V&V, or, more likely, their V&V teams were overridden by an executive.

(Bit like Boeing really. All worship the almighty dollar in the C-suite.)
Title: Re: The Great Windows Outage of 2024
Post by: Orior on July 19, 2024, 08:46:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 08:26:10 PM
Quote from: Mourne Red on July 19, 2024, 10:05:51 AMAs someone in IT support - It's been hell so far, Crowdstrike/Microsoft going to have big compensation claims against them

Dunno if it has much to do with MS.

Seems like Crowdstrike have either a bunch of cowboys running their V&V, or, more likely, their V&V teams were overridden by an executive.

(Bit like Boeing really. All worship the almighty dollar in the C-suite.)

Is V&V validation & verification?

I assume they also have a CAB - Change Advisory Board.
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 09:34:31 PM
Software testing is a very underinvested in function across the board. You would have to imagine this kind of software update is rolled out very regularly by a company, and probably product, like this.
Title: Re: The Great Windows Outage of 2024
Post by: RadioGAAGAA on July 19, 2024, 10:00:53 PM
Quote from: Orior on July 19, 2024, 08:46:57 PMIs V&V validation & verification?

I assume they also have a CAB - Change Advisory Board.

Yup - well - pedantically Verification & Validation 'cos verification should always come first of the two.

Crowdstrike seemingly have done neither well as both check gates should have caught this.
Title: Re: The Great Windows Outage of 2024
Post by: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.
Title: Re: The Great Windows Outage of 2024
Post by: Tony Baloney on July 19, 2024, 10:05:50 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board. You would have to imagine this kind of software update is rolled out very regularly by a company, and probably product, like this.
Complaceny combined with cost cutting dressed up us a productivity/efficiency/lean programme will be in there somewhere.
Title: Re: The Great Windows Outage of 2024
Post by: RadioGAAGAA on July 19, 2024, 10:07:27 PM
Quote from: Tony Baloney on July 19, 2024, 10:05:50 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board. You would have to imagine this kind of software update is rolled out very regularly by a company, and probably product, like this.
Complaceny combined with cost cutting dressed up us a productivity/efficiency/lean programme will be in there somewhere.

"Shareholder value"

Well, value your f**king shares today lads.


Uhh, actually, just looking - their shareprice has been diving since July 15th - so much so that todays disaster hasn't actually changed its general direction much.

What the insider trading f**k?!?!?
Title: Re: The Great Windows Outage of 2024
Post by: armaghniac on July 19, 2024, 10:09:08 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:07:27 PM
Quote from: Tony Baloney on July 19, 2024, 10:05:50 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board. You would have to imagine this kind of software update is rolled out very regularly by a company, and probably product, like this.
Complaceny combined with cost cutting dressed up us a productivity/efficiency/lean programme will be in there somewhere.

"Shareholder value"

Well, value your f**king shares today lads.



CrowdStrike : Software company
CrowdStrike Holdings, Inc. is an American cybersecurity technology company based in Austin, Texas. It provides cloud workload protection and endpoint security, threat intelligence, and cyberattack response services.
Stock price: CRWD (NASDAQ) US$304.96 -38.09 (-11.10%)
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.
Title: Re: The Great Windows Outage of 2024
Post by: ONeill on July 19, 2024, 10:25:13 PM
There'll be a big solar flare sometime soon that'll knock us for six. Get some cash in the attic.
Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 19, 2024, 10:26:37 PM
Ah sure as long as it gets here before trump gets back in that's grand.
Title: Re: The Great Windows Outage of 2024
Post by: thebigfella on July 19, 2024, 10:46:24 PM
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.


There is so many people working in technical roles that are incompetent full stop.

It's easy to blame managers and c suite but I'd say 50% (conservative estimate) of the tech resources I've come across on large scale programmes just are no where near good enough.
Title: Re: The Great Windows Outage of 2024
Post by: armaghniac on July 19, 2024, 10:51:44 PM
Quote from: thebigfella on July 19, 2024, 10:46:24 PM
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.


There is so many people working in technical roles that are incompetent full stop.

It's easy to blame managers and c suite but I'd say 50% (conservative estimate) of the tech resources I've come across on large scale programmes just are no where near good enough.

It is hard to get people. But surely Windows is partly to blame here, you can update a driver and so prevent the machine booting. A more robust system would sandbox the new driver in some way until it could be validated and only then add it to the system. I suppose Cloudstrike could have designed their updates along those lines, but the OS does not enforce it.
Title: Re: The Great Windows Outage of 2024
Post by: thebigfella on July 19, 2024, 11:03:28 PM
Quote from: armaghniac on July 19, 2024, 10:51:44 PM
Quote from: thebigfella on July 19, 2024, 10:46:24 PM
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.
Quote from: imtommygunn on July 19, 2024, 10:18:57 PM
Quote from: RadioGAAGAA on July 19, 2024, 10:01:42 PM
Quote from: imtommygunn on July 19, 2024, 09:34:31 PMSoftware testing is a very underinvested in function across the board.
Its also the one that suffers most from every other bit of the task sliding to the right.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.

There are so many people in the software business at management and cxo levels dictating what is going on without having the first clue about software of any description. The most important stuff falls by the wayside.

Share price was down to 291 earlier so they have done alright considering.

There was a big Reddit post from yesterday about how shit crowdstrike are... Just one post but still funny timing.


There is so many people working in technical roles that are incompetent full stop.

It's easy to blame managers and c suite but I'd say 50% (conservative estimate) of the tech resources I've come across on large scale programmes just are no where near good enough.

It is hard to get people. But surely Windows is partly to blame here, you can update a driver and so prevent the machine booting. A more robust system would sandbox the new driver in some way until it could be validated and only then add it to the system. I suppose Cloudstrike could have designed their updates along those lines, but the OS does not enforce it.

It's not though. Now it is hard to identify competent people in a market saturated with mediocrity.

Title: Re: The Great Windows Outage of 2024
Post by: imtommygunn on July 20, 2024, 08:08:37 AM
It is easy but in a lot of cases accurate.

Definitely where I work there's not much willingness to invest in technical talent.

The other thing is this is not a failure of 1 technical person it's a process problem from start to finish. There is probably technical and management and cost cutting etc etc in there so all at play.

Also windows as armaghniac says above and a long list really.
Title: Re: The Great Windows Outage of 2024
Post by: Milltown Row2 on July 20, 2024, 08:18:36 AM
I'd someone in with me yesterday, said he sells cloudstrike, it wiped off 3000 contracts, thinks they'll find it hard to recover.

But was reading there's not that many competitors? Would find that strange enough
Title: Re: The Great Windows Outage of 2024
Post by: RadioGAAGAA on July 20, 2024, 08:47:41 AM
Quote from: Milltown Row2 on July 20, 2024, 08:18:36 AMI'd someone in with me yesterday, said he sells cloudstrike, it wiped off 3000 contracts, thinks they'll find it hard to recover.

I'd be very surprised if they recover.

Would you trust them with any mission critical systems again?
Title: Re: The Great Windows Outage of 2024
Post by: armaghniac on July 20, 2024, 12:03:02 PM
Serious impact on flights for some airlines
https://x.com/US_Stormwatch/status/1814268813879206397