How NASA uses Big Data in practice
NASA’s Jet Propulsion Labs, which runs the day-to-day mission planning, has rebuilt its analytics systems around Elastic search, which now processes all of the data transmitted from the Curiosity during its four daily scheduled uploads.
Until recently, all of Curiosity’s operations were planned a day in advance, based on data received the previous day. This move to real-time analytics vastly speeds up the time in which decisions can be taken by mission control. NASA JPL data scientist Dan Isla told me, “This has been very transformational from an operational point of view. We can get very quick insights over multiple days’ worth of data, or even the entire mission, in just a few seconds, without delays. We can actually ask questions and get answers quicker than we can think up new questions.”
One application for the analytics is anomaly resolution. When it appears that there is a problem with the spacecraft, precise details of its operations can immediately be analysed to find out when this situation last occurred, and what other elements were in play at the time.
Isla tells me “Curiosity was designed to find out whether Mars was ever a habitable environment. During the course of the mission we’ve been able to confirm that yes, it was. The next mission – in 2020 – we’re taking life detection instruments and we need more capabilities, more instruments and we need to be able to do it faster. We can’t be spending hours analysing the data, so we need to take the Elastic search approach that we implemented fairly early into Curiosity and do that from day one.”
The technical details
Data transmitted from the Curiosity covers tens of thousands of data points, including every reading taken by its on board sensors, such as temperature on the Martian surface, atmospheric composition, and precise data on its own equipment, tools and actions.
The Elastic search program – which recently achieved its 50 millionth download – means that patterns and anomalies in the data can be spotted far more quickly.
Ideas and insights you can steal
It’s very telling that NASA’s next Mars mission is being built around the Elastic search technology from the ground up, which shows just how vital real-time analytics are to the team. In business, too, the ability to quickly get at insights and make decisions based on what the data is indicating can give a company the all-important competitive edge.
You can read more about how organisations are using Big Data to drive success in Big Data in Practice: How 45 Successful Companies Used Big Data Analytics to Deliver Extraordinary Results.
Bernard Marr is a bestselling author, keynote speaker, and advisor to companies and governments. He has worked with and advised many of the world's best-known organisations. LinkedIn has recently ranked Bernard as one of the top 10 Business Influencers in the world (in fact, No 5 - just behind Bill Gates and Richard Branson). He writes on the topics of intelligent business performance for various publications including Forbes, HuffPost, and LinkedIn Pulse. His blogs and SlideShare presentation have millions of readers.