Content-Length: 248964 | pFad | http://github.com/18F/analytics.usa.gov/issues/1432

F5 Missing Data for 11.6.24 and 11.7.24 · Issue #1432 · 18F/analytics.usa.gov · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing Data for 11.6.24 and 11.7.24 #1432

Closed
mm225022 opened this issue Nov 18, 2024 · 6 comments
Closed

Missing Data for 11.6.24 and 11.7.24 #1432

mm225022 opened this issue Nov 18, 2024 · 6 comments

Comments

@mm225022
Copy link

mm225022 commented Nov 18, 2024 •
edited
Loading

I have noticed that there is missing data for November 6 and 7 of this year

I have pulled data using the API (https://api.gsa.gov/analytics/dap/v2.0.0/domain/) for the following 3 domains and all 3 are missing data for these two dates:

The data that I get back for the period in November between November 4th and November 9th for www.medicare.gov, for example, is

 id       date            report_name  report_agency           domain           visits
 38926850 2024-11-09      domain                  <NA>         www.medicare.gov 383671
 39002639 2024-11-09      domain       health-human-services   www.medicare.gov 374258
 38926818 2024-11-08      domain                  <NA>         www.medicare.gov 614795
 39002625 2024-11-08      domain       health-human-services   www.medicare.gov 595694
 38859712 2024-11-05      domain                  <NA>         www.medicare.gov 543278
 38886907 2024-11-05      domain       health-human-services   www.medicare.gov 543707
 38778890 2024-11-04      domain                  <NA>         www.medicare.gov 612624
 38812149 2024-11-04      domain       health-human-services   www.medicare.gov 612624

The data for 2024-11-06 and 2024-11-07 is just missing.

Is it possible to get the data restored retroactively for those two dates for at least the three sites that I mention above?

Thanks
@mm225022
Copy link
Author

I wanted to link this to a prior issue #1095.

I also now see that, in addition to the two dates I wrote about in November, data is missing for December 3, 2024 as well.

@levinmr
Copy link
Contributor

levinmr commented Dec 20, 2024

Thanks for the report. We are investigating and will address the issue

@levinmr
Copy link
Contributor

levinmr commented Dec 20, 2024 •
edited
Loading

The application is currently in the process of backloading the missing data. It will take a few hours.

@mm225022
Copy link
Author

Thank you @levinmr . I can see that the missing data that I reported is now present.
However, when I ran this morning, it appears to be missing 2024-12-26 and 2024-12-27.
Could I trouble you to please look at that and update if possible?
Thanks

@levinmr
Copy link
Contributor

levinmr commented Jan 6, 2025

After some more investigation, I determined the source of the problem. The process which signals the loading of API data had too many actions happening in parallel and would sometimes run out of memory and crash before the API loading was triggered. This has been addressed and shouldn't happen in the future.

The data for 12/26 and 12/27 is in the process of backloading and should be present by EOD.

@mm225022
Copy link
Author

mm225022 commented Jan 6, 2025

Excellent. Thank you Mike!

@levinmr levinmr closed this as completed Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@levinmr @mm225022








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/18F/analytics.usa.gov/issues/1432

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy