Data from obdaac (earthaccess API) vs oceandata (oceandata.sci.gsfc.nasa.gov)

[adlinhamdan]

Hi,

I am writing a dynamic polling script to get as recent data as possible. I just want to confirm my observation that if I pull all granules from today, the earthaccess API will show less granules and might take time to be fully updated. For example, at 20 UTC, I got 45 granules (PACE_OCI_L2_BGC_NRT) for Feb 13 using the Earthaccess API, but at the same time if I go to the website https://oceandata.sci.gsfc.nasa.gov/directdataaccess/Level-2/PACE-OCI/2025/13-Feb-2025/ it will show recently updated granules and i found 97 of the same file type. Is there a way I can access more recent data using the Earthacess API?

[mfisher87]

You are correct in this observation! earthaccess uses the NASA Common Metadata Repository (CMR) to find data. It would be impractical for earthaccess to understand file and directory structure for every dataset, so we rely on data ingest teams to populate CMR when they release new data. This is not always immediate! Based on the OBDAAC contact page, I'd reach out on the EarthData Forum if you would like to inquire further.

@itcarroll any context you can add?

[itcarroll]

I do not know that this is contributing, but the OB.DAAC is in the midst of reprocessing Version 2 to Version 3 of this product. It could be increasing the lag. (I will find out tomorrow what the DAAC expects for the lag.)

In case it's not clear, search.earthdata.nasa.gov uses CMR too.

OP may be interested in the new CMR subscription service.

[asteiker]

Since @itcarroll mentioned CMR Subscriptions, another resource could be the PO.DAAC Data Subscriber: https://github.com/podaac/data-subscriber/tree/main?tab=readme-ov-file. This tool should support any NASA Earthdata collection including data from PACE.

[itcarroll]

The OB DAAC doesn't control the whole process for populating granules and metadata on Earthdata Cloud and CMR. Often times the lag is negligible, but clearly not always. Best I can say is that if your requirement is to have less than a couple hours of lag, then you will need to query the OB DAAC filesearch utility. See the "Using the API" tab at https://oceandata.sci.gsfc.nasa.gov/api/file_search_help/.