Leap second hits Qantas air bookings, while Reddit and Mozilla stutter

The Guardian
The addition of an extra second between Saturday and Sunday to account for the slowing rotation of the Earth affected flight check-ins in Australia, and hit popular websites including Yelp and Foursqure

The Amadeus airline reservation system was disrupted for more than two hours on Sunday, as were a number of high-profile websites including Mozilla, Reddit, Gawker, LinkedIn, FourSquare and Yelp as the "leap second" introduced as the day began confused web servers around the world.

In an echo of the "Y2K" bug that was expected to – but didn't – befuddle computers around the world on 1 January 2000, travellers in Australia, where the effects were felt first as the leap second was introduced, saw more than 400 Qantas flights delayed by as much as two hours. Staff were forced to switch to manual check-ins.

The problem came after a weekend when a number of US-based services which rely on Amazon's cloud services were knocked out for hours due to power failures in the Seattle company's servers after tropical storm Debby hit the US on Saturday.

The addition of the extra second occurred as Saturday turned into Sunday in London – which was around midday on Sunday in Australia, and at 5pm Saturday on Pacific Standard Time. That meant that some web systems found their own clocks at odds with external ones.

Amadeus handles more than 3m bookings and 1bn transactions per day for most major airlines worldwide, through a data centre in Germany.

The company said that it was investigating the cause of Sunday's crash and would take any appropriate steps to avoid the situation reoccurring. "This incident was caused by the Linux bug triggered by the 'leap second' inserted into clocks worldwide on June 30th," said an Amadeus spokesperson. "We deeply regret this incident and want to apologise for any inconvenience it may have caused to our customers or to travellers and tourists. We are now investigating how we can enhance our ability to detect and address such bugs in advance. We take any systems disruption very seriously, we have always valued our reputation for reliability and we are determined to do everything that is appropriate to provide a reliable service in future."

The outage is the second this year, following one in January.

The leap second was introduced at midnight UTC, or Universal Coordinated Time – roughly the same as Greenwich Mean Time (GMT) – as Saturday turned into Sunday.

The intention was to keep UTC on which atomic clocks run coordinated with the rotation of the Earth, which is gradually slowing down due to tidal effects from the moon.

The cause of the glitch arose from sites' use of NTP, the Network Time Protocol. Although a huge number of computers rely on NTP to keep accurate time, it's still unclear why some sites but not others were affected.

For Mozilla, which provides the Firefox browser and the discussion site Reddit, the problems were similar. In a Mozilla bug report titled "Java is choking on leap second", Eric Ziegenhorn, a site reliability engineer for Mozilla, said that the Hadoop open-source database platform was having problems.

"Servers runing Java apps such as Hadoop and ElasticSearch and java doesn't [sic] appear to be working," he wrote. "We believe this is relate to the leap second happening tonight because it happened at midnight GMT."

The solution in many cases was to reboot the servers, other people on the thread discussing the problem said. Mozilla later pushed out a fix.

Reddit said in a message on Twitter that the extra second was causing problems with its Java-based server, while the Gawker website was unreachable for 30 minutes, its chief technology officer Tom Plunkett told CNet. StumbleUpon, an aggregator system, also reportedly suffered problems.

The failure in Amazon's cloud systems affected a wide range of companies, including the photo-sharing service Instagram, the "pinning" service Pinterest, and Netflix.

Adding leap seconds – and dealing with the effects – will continue to be a challenge for scientists and engineers, because the Earth-moon gravitational pairing will see our planet's rotation slow down as they orbit their centre of gravity.

Given billions of years, the Earth and moon will be locked in a mutual orbit in which they always show the same face to each other, putting the moon in the same place in the sky forever.

The Earth's rotation has been slowing down since the moon was formed; in the era of the dinosaurs, a day was about 23 hours long, and now the days are getting longer by about 1.4 milliseconds every century.

A single rotation took exactly 24 hours in 1820; since then the "day" has been growing longer.

EGYPT NEWS