How can we enhance our metrics reporting?

[asteiker]

I have been thinking about key performance indicators that quantify the growth and value of earthaccess. We have a great start with GH response metrics (e.g. #934), and as part of v0.9.0 we added user-agent to track requests to Earthdata Login (see #443). Here are some of the questions I'm interested in:

• What percentage of overall distribution across ESDIS data originates from earthaccess and how is this broken down by:
  • collection
  • mission
  • DAAC
  • Number of requests vs volume
• What percentage of earthaccess distribution is accessed via s3 vs download vs HTTPS streaming
• What are the total CMR search hits originating from earthaccess, and percentage compared to all CMR search hits
• etc.

Tagging a few of our DAAC maintainers who may have access to our Earthdata Cloud ElasticSearch metrics: @itcarroll @danielfromearth @battistowx @jhkennedy. I am familiar with Kibana but would need additional guidance to start gathering this info, and I'm not even sure whether we have everything set up to pull these types of metrics. We may also need to think about metrics from EMS for on-prem data. I'd love more discussion on what other metrics we may find useful to highlight the value and usage of earthaccess as the primary python-based search and access tool for NASA Earthdata.

[JessicaS11]

Not helpful on the ESDIS data side, but icepyx has a few GH workflows that pull (via a scheduled cron) PyPI and conda-forge download data (one of those only keeps historical data for like 90 days) and plot them. It's not fully automated (the plots and data are automatically updated on a dedicated "traffic" branch that I have to manually merge before each release and then recreate the branch), and there may be better GH actions available now too, but it has been useful for always having an up-to-date plot of download stats as part of the docs. I know earthaccess already has some of these types of metrics being tracked, but can't recall which ones.