Redesign of the reporting section

[sei-bstein]

Members of the team have agreed that the Reports page in the Admin section of Gameboard would benefit from usability improvements and new features. Some initial ideas are:

• Clearer names and descriptions for existing reports
• The ability to parameterize relevant reports and to adjust those parameters while viewing them
• Infrastructure that makes adding new reports easier
• A customized presentation, per-report, of its data and features that best support what it communicates
• CSV export should be preserved (potentially to XLSX/PDF as well?)

Additional requirements from project leadership:

• Registration numbers per board per day (more likely to be a dynamic period of time with a day default)
  • NOTE: We'll need a solution for historical data. SQL server has temporal tables, but PGSql only supports this with an extension, and it's unclear how that will interoperate with EF
• How many unique users signed up for competition each year
• User/track participation
  • How many people engaged only in a specific Track (Teams, A or B)?
  • How many did two of these, and which ones?
  • How many did all three?
• Per board metrics:
  • How many users per sponsor enrolled vs. actually played
  • How many teams from each sponsor enrolled vs. actually played
• Per challenge metrics:
  • How many partial vs. complete solves
  • Average points scored
  • Average time to complete solve
  • Stretch goal: among users who have the same partial solve (by questions), what was the average time?
• App usage metrics
  • Challenge deploy count
  • How many distinct users from each sponsor have an account?
  • TBD: how many actually play(?)

[sei-bstein]

Additional ideas for reports from ownership:

users report (?)
How many users does the site have?
How many of those users have ever enrolled in a game?
How many users have ever started a session?
How many users have ever started a challenge?

engagement report (?)
How many sessions have ever been started?
How many challenges have ever been deployed? (note: data from ArchivedChallenges may matter here)
How many sessions per user? -- E.g., 20% of users have 1 session, 15% of users have 2 sessions, 30% of users have 3 sessions, etc. (Pie chart?)
Same as above for challenges per user.

[sei-bstein]

Summary of questions the revamped reports may address (and how well they currently answer these questions):

Question	Report	Answered?	Notes
How many users signed up for the competition each year?	Players Report	Y	This report shows creation date and can be segmented with filters like dates and "Series" (competition year)
How many users registered per game per day?	Enrollment Report	Y	This report doesn't do aggregations (like averages), but its data can support that kind of analysis.
How many users played only in a specific track?	Enrollment Report	N	We can report on numbers for a specific track, but we don't currently support an exclusive "only this track" filter
How many users played each of two tracks (for all combinations)?	Players Report	N	We don't have "And"-style aggregation for multi-select filters like tracks
How many users played in all possible tracks?	--	N	We don't have an "And" style aggregation for multi-select filters like tracks
How many users per sponsor have an account?	Enrollment Report	Y?	We don't have an aggregation of sponsor/users, but the data is in the CSV export, and the report can display a single sponsor's user count
How many users per sponsor enrolled/played?	Enrollment Report	Y	This report doesn't aggregate by sponsor, but the data you can export from it could be used to answer this.
How many teams per sponsor enrolled/played	Enrollment Report	Y	Same as above. We don't allow filtering by team vs. individual, but the export contains each player's team Id.
Partial vs. complete solves	Enrollment Report + Practice Report	Y	--
Average points scored per challenge	Challenges Report	Y	--
Average time to complete solve	Challenges Report	Y	--
Among users with the same partial solve, average time?	--	N	This'll be exciting
Overall challenge deploy count/trend	--	N	This seems like it'd benefit from a line chart style viz
How many users does the site have?	Players Report	Y	--
How many users have ever started a session?	Players Report	?	Right now we're reporting challenges deployed/finished, but that is technically different than a session. Does the distinction matter?
Per-user session percentages e.g. "20% of users have 1, 15% have 2, etc."	--	N	--
Per-user challenge percentages e.g. "20% of users have solved a challenge, 15% have solved 2, etc."	--	N	--

[sei-bstein]

Additional points to address from internal review:

• Report for Prac?
• Distinguishing between team and individual performances on challenges (Challenges Report)
• Event Horizon data as report (Reincarnate "Event Horizon" feature #110)
• Challenge Report - use date range for deploy time
• Surface player-level info to players outside of the reporting section

[sei-bstein]

Todo: evaluate old reports to ensure necessary ones are ported forward.

[sei-jbooz]

Reports that currently exist:

1. Enrollment Report - view a summary of player enrollment
2. Practice Report - view a summary of player participation in the Practice Area
3. Support Report - view a summary of report tickets

Questions about the current reports:

1. Does the enrollment report show users that have an account but are not enrolled in any competitive games?
2. In this comment is the mentioned "Games Report" the "Enrollment Report" above?

Additional Reports we may want:

1. Overall usage report - view a summary of how many challenges (practice and competitive) have been deployed, how many users have accounts, how many new accounts over a period of time. Stretch for this: how many hours of use does the site have (maybe we add up cumulative time for each unique team to get to this number showing hours of challenge play time?)

[sei-bstein]

Request captured in another issue: average practice challenges completed per user (globally?)

[sei-jbooz]

Related to #209

[sei-bstein]

Updated our requirements table in this comment

[sei-bstein]

This is a good place to track the updated Reporting effort, but I moved it to "Blocked" in the project because I don't expect us to have any bandwidth for it (except for minor issues) until late April.

[jbooz1]

Update from Stakeholders:

We should port the old "Feedback Report" to the new reporting interface.

#465

[sei-bstein]

Closed in favor of individual items as needed.