Roadmap

Input capture widget

Taking the data from the database that is now being ingested as part of the upload process, display a timeline of the user inputs in a similar style to guitar hero (potentially). Need to account for controller and keyboard inputs

Widget based UI

Add the missing UI elements from the composite video player including: Remove the need for a composite video player and instead have the UI play in-sync so users can follow along with the video player, seeing other UI elements update as the video progresses (similar to how transcription works currently). This will speed up the capture process further and remove risks of issues with the composition compilation process. Feature Specification: TMROADMAP-50 - Synced player & standalone event widgets - Product Ownership - TestMod Confluence

Event Classification, Description & Re-Naming

This ticket is focused on improving the way users classify and share their events. The first stage is to allow users to add text descriptions to an event in a similar manner to Jira. The second is to allow classification of the event by assigning a label to the event. By default, users should be able to classify events by:

Self-Serve Admin

Build out session and event state reporting on Project for admins Session states = active and closed//complete, surface if session has error State 5 = composition failure Session states = active and closed//complete, surface if session has error

Edit Transcripts

Session Review and Edit Screen

Improve the user experience of the app’s event system by utilising the Session Complete screen. This will transform this screen to become a Session timeline where users can easily review the events they have captured, edit them and add new clips after the session has finished. This also includes work to increase visibility of events inside the session management screen so users can see the status of any events triggered whilst recording. Spec: TMROADMAP-66 - Session Review and Edit - Product Ownership - TestMod Confluence

Voice Annotation

Currently, users are unable to add voice notes to events after they have created the event. This means to add commentary to the bug, they have to have their microphone on for the full recording and call out everything they see, which isn’t realistic. This issue also impacts our ability to use LLM’s to automatically generate tickets based on the transcript as it isn’t often there. Only 12% of our clips so far have microphone data attached. Users will be able to add their own voice notes to events and control where they start playing. This will allow viewers of the clips to hear the voice notes over the top of the audio of the recorded video. These voice notes will also be transcribed automatically.

User Settings & Project Management

Review the user structure and way that users manage their own data and projects / companies they have permissions to access. This includes making the following revisions to the user model: Also requires the creation of a company admin user role who can: Project leads (Managers) will then need to be able to:

Text, shape & drawing annotation

Inside the app, and on the dashboard, give users to add notes or highlight any specific parts of a captured screenshot.

Console Feature Parity

Add support for user input and performance data capture on consoles.

Automated clip capture

We will eventually have custom filters for the Dashboard for performance. Is there a possibility of capturing a clip automatically if those filters are met when recording? Example: I have it set to alert me on the dashboard if FPS drops below 30. While recording, have topmod automatically capture a 1 minute clip (say.. 15 seconds before, 45 seconds after) if FPS drops to 30 and flag that on the dashboard automatically. Could do this for memory as well, say doing a soak test, if memory keeps increasing, flag the performance data as a potential memory leak and upload to dashboard.

Capture isolated audio channel

Have had feedback from a tester that they would like to be able to select which audio channel is captured as part of the recording. Currently, if they are listening a piece of media on the device when testing, to record the audio of the game, they either have to:

Adjustable recording settings

Give user ability to: See image attachment for a time where image wasn’t legible. Example here - TopMod QA - Event View - F83C3235-E422-419B-A140-E9F2C93F76D1 - If you look at the blue and red text floating in the game. Worth keeping this as a reference for future tests.

Capture VRAM

Keyboard Shortcut support

We have currently removed the keyboard shortcut for auto-clipping. This functionality should be in there but as its for our more advanced users, it should be opt-in. Therefore, we need to add it back in but only if users set the key binding for it. To manage this, we will need to update the settings screen to make sure this is clear and to recommend the key binding we have currently. If the user then chooses to enable it, the following should happen when they press the shortcut:

Event Approval Process

Give Managers the ability to see a review queue of events flagged by testers on a project and then mark individual events as approved/denied. Store the average approval rating for each individual user and allow test leads to filter through events based on this. Also build an event viewer page for clients to review approved events in. Allow testers to label when something is ready for review.

Advanced Search

Create Advanced Search Page and allow users to search for Sessions/Events where: Use natural language and dropdowns to allow for filtering: "Show me all events where CPU went over [x] for this game" Save searches in sidebar for quick search

Jira Integration

Jira detection for two reasons:

Advanced DXDiag handling

Show DXDiag contents in a more friendly way than just displaying a whole file and assign to user so that filters and other searches can take place

Search Audio transcripts, summarize and auto-tag

Search by text on the screen to get instant clips showing specific experiences across all players, this helps to build confidence at pace when preparing key findings to share with your team. Detect when users say something like "Ah that was shit" and then highlight and allow it to be searched outside of the Session. Should detect any expressive terms Automatic summarisation into bug ticket format, potentially via a LLM

User driven thresholds / Alerts System

Ability for users to set the thresholds and then see alerts across their performance data when breached On session, project and other pages, integrate violations of thresholds and email notifications to project leads.

Enhanced Performance Data Analysis

This feature will likely be driven by a combination of various elements but will feature:

Workspace Support & Dedicated Project Pages

Transform the way collections work across the application so that we have: This will also be coordinated with a refreshed Project Explorer and surfacing of recent sessions across the dashboard.

Bookmark System

Categorised Verbatim's From Transcription

As part of our TopMod training we ask for TopMod Leads to categorize and sort Transcriptions ”For categorizing verbatims, it’s important to review the test plan/focuses. For example, if you have a test with a focus on Difficulty levels, making sure to pull verbatims on the Difficulty and correctly categorize them as “Difficulty." This isn’t to say that you can’t pull and categorize verbatims that are not a part of the focuses of the test, but the focuses are the most important part. Correctly categorizing takes a bit of practice, but when in doubt, ask others’ opinions in the leadership channels!” Idea/Proposal: Allow PMS to assign categories to individual (where we allow edit) verbatims in TopMod events/sessions then pull a CSV of all categories and verbatims within a project. This would then be aggregated together inside the dedicated Project page for review.

Expanded Session and Event Information

* Add the ability to add labels to events for easier segmentation in the performance dashboard * Add the ability to add version / build information to the Sessions (in app and in dashboard) * Add the ability to capture and attach custom logs * Add the ability to capture what platform user is reporting from and display in dashboard (PC, Xbox, PlayStation, Switch, Mobile) * Add session activity i.e last accessed, number of times accessed

Password Protected Share Links

Bug Submission and Review Process

Mobile Feature Parity

Add support for performance data and user input data capture.

Outward API

Subtitles Overlay

Audit Trail

Focus Group Mode

List all "Live" Sessions being recorded right now and allow manager/admin to jump in and view live stream. Should allow for two-way communication and multiple people (Less than 10) to view at one time Should showcase data streams from testers and allow annotation on stream

Third Party Tools Integrations

Slack, email etc…

Chart Overlays

Headless Client

Multi-client recordings

The ability to say, record 3 screens at once.

Desktop capture

Consider adding full screen capture (although it means limited data capture taking place) as will be good for fallback when app connection doesn’t work

App Notification system

Persistent push messaging in app so we can notify clients to update the app, known outages/issues, etc. when they login. Also use to inform the user of recordings being uploaded. If possible, tie into Windows notification system so user can receive messages outside of app.

Automated Accessibility Testing

Stream recordings to server

Guided Recording Session

Allow project creators to set what recording data is being captured and force inputs to be set to on and prevent user from turning them off. Also allow the project creator to set a series of instructions for the test session that will appear when the user clicks “Start Session” in the application. Finally, allow all projects to have descriptions/guidance added to them that is accessible inside the Session Management screen of the Recording application.

Engine SDK

Developer integrates with TopModQA using localhost API calls to flag which scene the user is in. We have eye tracking implemented, along with input analysis, so analytics on how long it took the user to complete a section, how many buttons they pressed, where they were looking can all be generated.

AI Bug Ticket Creation

Taking the available data from classification, description and transcription of clips, allow users to automatically generate a Jira bug ticket with links to the TopMod session already populated.

See Performance Data mid-recording