Global BSOD

[Journey]

Watching the mainstream media report this is hilarious.

 

[d_brennen]

We had CrowdStrike for a couple of months last year as part of a security assesment. Glad it's gone today!

 

[GGizmo]

But it's a Crowdstrike issue. Yes affecting Microsoft servers but it's not their doing.

From what I understand there are two issues going on at the same time. They might be connected. But they need to be solved individually.

Unless I'm wrong about that?

 

[alec]

More and more big companies releasing explanations like
"Third party software/Crowdstrike is to blame for this outage, blah blah"

No. YOU are to blame for installing kernel mode malware across critical systems on our network!

 

[Noxia]

Interesting timing, just started watching 'Leave the world behind'.

Thankfully RIAT still going on.

 

[Journey]

Unless I'm wrong about that?

You are wrong, the media reporting before they had all the facts.

 

[Felon]

oh dear,

So yesterday Crowdstrike published an article about how skipping code reviews is bad, https://www.theregister.com/2024/07/18/security_review_failure/

How embarrassing

 

[GGizmo]

You are wrong, the media reporting before they had all the facts.

So it's purely a cloudstrike problem?

 

[kindai]

Anyone slightly concerned how reliant we are on all this IT infrastructure yet how easy it seems to be to disrupt?

This is why moving everything to "the cloud" is an absolute disaster waiting to happen is a terrible idea.

Particularly when so many services all end up at the same end provider.

 

[explicit4u]

Good thing for us it happened on Friday, our office is usually at >5% capacity. VPN and DCs have gone down so no doubt everyone working from home will be out sunbathing

 

[visibleman]

Luckily i don't deal with CrowdStrike but i know a few IT guys that do and they're struggling with bringing up the entire org's. A rubber ducky would be extremely useful i think...

The fix:

How to fix the CloudStrike Windows BSOD issue​

Fortunately, CloudStrike has since announced at 2:30 a.m. ET that it has identified the update causing the issue and rolled it back. The company also offered a workaround for anyone having problems:

1. "Boot Windows into Safe Mode or the Windows Recovery Environment
2. "Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
3. "Locate the file matching 'C-0000029*.sys', and delete it.
4. "Boot the host normally."

CrowdStrike Falcon is mainly used by businesses but some allow their employees to use it.

One liner CMD -

del "C:\Windows\System32\drivers\CrowdStrike\C-00000291*.sys"

If you've got Bitlocker enabled then you might run into issues although could try and force safeboot through bcdedit.

Microsoft will end up collapsing at some point unless they don't start up'ing their game.

This specific issue isn't MS but rather CrowdStrike.

 

[throwaway4372]

Ooft.
Do I buy some crowdstrike shares.. Or is this reputational damage?

the premarket price is just the price you could have bought at in april, it would have to fall a lot more (which it might) to be seen as a discount imo, I'd say wait and see. My prediction is it should be possible to get it for under 200, or even 100 because imo it's massively over-valued.

 

[Glanza]

Glad we didn't proceed with Crowdstrike looking at what's caused the issue and the fix isn't easy for people to roll out remotely.

 

[Journey]

So it's purely a cloudstrike problem?

It causes your MS server to go into a boot loop due to an update CS pushed, it is a CS caused issue.

 

[molinari]

So it's purely a cloudstrike problem?

Yes. Simply renaming the Crowdstrike folder (or deleting the file as posted) and then rebooting the server, fixes it. We have had about 20 of 200 servers affected.
I think the reason Microsoft was mentioned specifically, is becuase it isn't affecting Linux servers with Crowdstrike agents installed, only MS servers.

 

[visibleman]

Probably a DEI hire at Crowdstrike.

 

[GGizmo]

This is why moving everything to "the cloud" is an absolute disaster waiting to happen is a terrible idea.

Particularly when so many services all end up at the same end provider.

I think the advertising terms "the cloud", or back in the day "cyber space" are deceptive terms and people get a false sense of security.

 

[Goksly]

Microsoft will end up collapsing at some point unless they don't start up'ing their game.

Have you seen their share price? They aren't going anywhere

 

[Skynet5]

Meant to be flying back from Mumbai tomorrow with a connecting flight from Dubai Sunday morning.

Fingers crossed

 

[Josh]

Busy morning for some techies, thankfully all my internal infra is unaffected but a lot of clients are suffering as are their 3rd party support teams! Imagine being the tech responsible and walking into "P1 - The World is down" ooof.

Pretty scary tbh, Die Hard 4 fire sale come to life somewhat.