Weather v TechnologyAdded : Thursday at 13:45 Technology is great... When it works.
But you may have noticed a few glitches over the past few days on our site as well as others which use the GFS ► data or the GFS ► Ensembles. Data missing, data late etc. The reason has been down to a rather major data issue at NCEP which run and produce the GFS ► model.
You may ask why we use the GFS ► model? It's a great model to be honest. It's free, plenty of weather variables and the accuracy and timesteps allow us to produce our forecasts with plenty of options and pretty charts for you.
But, that is when everything is going well. A couple of days ago we were getting intermittent errors connecting to NCEP. Intermittent are the worse types of errors as you need to figure out a new layer of "what is causing this" before applying the fix.
We managed to get hold of the GFS ► and the Wave Data, but the ensembles are a different kittle of fish.
The problem comes from last year when NCEP started to restrict data downloads to 100 requests per minute. Sounds like loads right? But when you have 30 model runs in the ensembles as well as a control run and timesteps out to T+384 in 3 hour intervals, so we would need to download around 11,000 files to create the forecasts.
If we are restricted to 100 a minute then, well, you can do the maths. It doesn't make sense to even try to get the files as they will be out of date and ready for another run by the time we finish. Plus, that is just the ensembles, we also need to get wave data and a few other things too!
The weather industry is unique though, we work 24/7/365 and there is no break in the updated forecasts becoming available. Sometimes things go wrong and we bet you don't even notice. Sometimes they go wrong and you do.
But then sometimes, we get major data outages which leave us juggling between what data we need and what to discard to stop our own servers from blocking up with the data backlog.
Hopefully, NCEP will be back to normal later today, but we will (today) be building a new redundancy system which isn't so reliant on NCEP.
As they say, once bitten, twice shy, but just a little overview of all the things which go on behind the scenes to create the pretty charts and graphics.
GARLAND |