Five Free and Reliable Weather Data Sources
High quality data to improve and augment your modeling processes
Weather influences human decision making in ways from the obvious to the subtle and unexpected. Energy providers need the data to make sure power throughput will meet demand when thousands of AC units on their grid turn. Demand for Citibike rides in New York City drops dramatically when it starts raining. Retailers even use weather data to target cold and flu medicine advertising at consumers during unseasonably cold weather.
A vast amount of weather data is collected by governments and academic institutions, meaning that when it comes to building analyses and models, the data you need is often available for free. This is particularly true in the US where data collected by the National Oceanic and Atmospheric Administration (NOAA), which includes the National Weather Service (NWS), is public domain.
The problem with much of this high-quality free data is that it's buried in hard to find, antiquated hosting areas, FTP servers, or two pages deep into Google behind a whole slew of private companies. For example, much of NOAA's data is only available through a shopping cart process where you may need to know the exact datasets you want before you even start looking. The difficulty in accessing these data means your easiest options are usually private data providers with APIs and high search rankings. Those sources can be great, but typically have free tiers with prohibitively strict rate limits, historical time limits, resolution limits, etc. So, to help you in your data search, I've put together five high quality and reliable data sources that are, most importantly, free.
National Centers for Environmental Information (NCEI)
Datasets | Climate Data Online (CDO) | National Climatic Data Center (NCDC) (noaa.gov)
NCEI, under the National Oceanic and Atmospheric Administration (NOAA), offers access to one of the most significant archives for environmental data on Earth. They provide comprehensive information on atmospheric, coastal, oceanic, and geophysical parameters. You can download data for historical weather patterns, paleoclimatology, solar events, and more in various formats.
If you follow the datasets link above, a great starting point is the Global Hourly Data under Legacy Applications. You'll be able to retrieve hourly data for most major airport-based weather stations globally, and every official station in the US (mostly but not always airport-based). If you're trying to augment data for a specific location, start with the nearest airport.
Another interesting dataset is the storm events database, which provides not only information about the likes of tornadoes and hail, but recorded outlier events such as snow in Florida, and weather-related events like avalanches that resulted fatalities.
Longer-term data such as Climate norms are available under Climate Data Online, and particularly useful for seasonality-based projects. In the same section you can also see Weather Radar, which is reflectivity data from NWS radar installations. The radar data is your best option if you're looking for something like hyper-local rainfall, and for better or for worse, will pose you a significant technical challenge in terms of making it usable for analysis in code.
As mentioned earlier, NOAA data is public domain so anything you collect here can be used for any purpose including commercial use.
The European Centre for Medium-Range Weather Forecasts (ECMWF)
The ECMWF is an independent organization based in the UK and supported by various European states. It provides medium to long-range weather forecasts and allows free access to its datasets through the ECMWF Web API. Their ERA5 dataset is especially valuable for climate research, providing detailed information about the atmosphere, land surface, and ocean waves from 1979 to present.
Unlike the NOAA observation data, ERA5 is what's known as "reanalysis data". In contrast to observations, reanalysis is essentially a model interpolated view of the weather across the modeled region. For ERA5, the data is global with a 0.25 degree resolution. Every data point location has multiple vertical levels available, so you can create vertical profiles of the atmosphere at any given time since 1979.
Helpfully for us data folk, these data are available through an excellent API that accesses the ECMWF Copernicus data store (CDS). To use the service, you need to register and get yourself API keys. A pro tip for using the API is to download data in NetCDF format and use the xarray Python module to read it into a pandas dataframe.
The ECMWF is high up this list because it's such a high quality and well-regarded source of data. However, you must be careful in understanding the terms of use for the data they provide as they don't operate under the public domain like US government sources. Of the publicly accessible content, some items are subject to their own terms of use. Such items either carry an indication of who controls the rights in the item and, in some cases, a link to a click use licence, or restrictions concerning their use are listed in image credits. The open data page is a good starting point, with summaries of available data, delivery means, and clear links to licence requirements.
National Weather Service (NWS)
If you're in the US and you've ever had a weather warning flash across your TV or pop up on your phone, it was a human weather forecaster at the NWS who issued it. If you need access to current data, e.g., you have a model or dashboard operating in real-time, this is where you'll want to start.
The API provides access to current conditions like temperature and precipitation, forecasts, and wave heights, for the entire United States and territorial waters. More limited reporting of just current conditions is available globally. Documentation and access details can be found on the NWS's official website, linked above.
As with the NCEI data, this is public domain and can be freely used for any purpose, however, be careful not to fall foul of rate limiting.
Open-Meteo
Free Open-Source Weather API | Open-Meteo.com
Open-Meteo is a collaborative project that aims to provide open, free access to weather data. While not a government source, it's included here because of its dedication to making meteorological data freely available.
They offer an API that provides access to numerous datasets, including forecasts, observational data, historical data, and radar images. The API supports multiple programming languages, making it a good choice for integrating data into applications.
You can use the API for personal and academic projects, but not for commercial ones.
The Japanese Meteorological Agency
Japan Meteorological Agency | Climate of Japan (jma.go.jp)
The Japan Meteorological Agency (JMA) provides weather, climate, and also seismology data free of charge. The available data isn't of the same granularity and depth as available through NOAA or the ECMWF, but provides the best source of local weather and climate information for Japan.
Of particular note, the JMA's Himawari weather satellite is one of the best in operation today, and provides extremely high-quality imagery of Japan, South East Asia, Australia, and New Zealand and the Pacific Islands.
While the JMA doesn't have a dedicated API, datasets including observations, forecasts, and warnings in a range of formats, including text, XML, and graphical formats, are available at the link above or in relatively easy to find areas of the website. If you start digging deeper, you'll probably need to use translation as much of the information is in Japanese.
Hopefully these sources are helpful, and you can find what you're looking for. Good luck with your projects!
