Funny thing is the preprod system wasn't any better, I sent a job there to avoid the wait, but the job started processing right away, and took three hours to complete.

On Tue, Jul 9, 2019 at 1:52 AM Michelle Devino <[log in to unmask]> wrote:
The batch I submitted at 2:26 p.m. did not complete until 6:46 p.m., just after Brian's batch. As was mentioned by both Marcia and Brian, four hours is a very long time to wait for a single batch/job to process, only to sometimes find there is a silly little error that requires a tweak and a resubmit. I did write the AOE support email group this afternoon/evening to ask about the processing speed. The processing time makes it almost impossible to complete the data collections during the regular work day. Processing times at night are significantly faster, but then you have to battle the nightly processes that bog down batches around midnight.

For comparison, I uploaded a batch at 1:29 a.m. this morning that had MORE records than the batch submitted this afternoon. The batch completed at 1:40 a.m. The job took just 11 minutes compared to 4 hours and 20 minutes, with the only real variation being the time of day the batch was uploaded.

As more and more districts begin utilizing the nightly unofficial data collections via edFusion over the coming months, the processing times are likely to only get worse as traffic increases. I hope the AOE will be able to work with edFusion to improve the processing power/speed so that we can all benefit from the future possibilities of automating the State Reporting process.

Michelle M. Devino
Data Manager
South Burlington
[log in to unmask]

Bob Wickberg
Technology Coordinator
Brattleboro Union High School District # 6


Search the SCHOOL-IT Archive

Manage your Subscription to SCHOOL-IT