Klarity Capacities
Capacities designates everything a user is able to achieve using Klarity.
Users
We identify different types of users :
- <safenai_member> : we are all tech ;
- <tech_user> :
- a tech from a partner with support and minimal documentation
- can also be a <safenai_member>
- <business_expert> : final customer business_expert
- <customer_other> : TBD
Version
- POC : June 2024
- MVP V1 : October 2024
- MVP V1.1 : December 2024
- MVP V2 : September 2025
- V1 : December 2025
- V2 : December 2026
- Target : Not planned, the content of the target must be prioritzed and distributed among the different versions.
Proposal
Proposed capacities must be discussed before they can be validated for integration and therefore, pushed into the backlog.
User-Journey
Generic User-Journey
[ ] <safenai_member> can push and store its artefacts and metrics on KS
- MVP V1.1
- [KS] [KC]
[ ] <safenai_member> can push and store its datasets set on KS
- MVP V1.1
- [KS] [KC]
[ ] <business_expert> user can provide the intended purpose and ROI elements (metrics) so that the corresponding artefacts can be created from KC.
- MVP V1.1
- [KD] [KC] [KV]
In MVP V2 the artefact creation is carried out by <tech_user> using KC.
[ ] <safenai_member> can locally create and manipulate artefact before delivering configuration that will generate the final ones to the cloud instance of KD
- MVP V2
- [KD] [KS] [KC]
Local instance KC and KV
Push from KC to Cloud KD will push data to KS and PCIV to KD [LCA not sure to understand][ ] <tech_user> can store its models set on KS
- MVP V2
- [KS]
[ ] When working locally, KD Front will use local storage system
- MVP V2
- [KD]
Currently, it is required to push the data in the front-end local repository and to rebuild the docker image which is a problematic workflow
[ ] <tech_user> can instanciate a blueprint to generate configuration and specific code in KC project
- MVP V2
- [BP] [KC]
./klarity folder in UC repository root to control KC and repo content
[ ] <safenai_member> can locally develop metrics and artefacts for a specific blueprint
- MVP V2
- [KD] [KC] [BP] [WB]
Requires to define the proper python environment : package, configuration, etc.)
Requires a set of tools to facilite the manipulation of the artefact/files[ ] <tech_user> can locally work and develop a use-case for a specific blueprint
- MVP V2
- [KV] [KC] [BP] [WB]
Requires to define the proper python environment : package, configuration, etc.)
Requires a set of tools to facilite the manipulation of the artefact/files[ ] <tech_user> can start a working environment with Jupyter/VSCode/...
- MVP V2
- [KS] [WB]
[ ] <tech_user> can use a CI to build and push artefact from WB/KC to KD
- MVP V2
- [KD] [KS] [KC] [WB]
using configuration and code use to test locally the metric / artefact generation, the CI will build "accountable artefacts" and push them as a versionn in klarity dashboard]
[ ] User can launch computation from the workbench over a blueprint instance of use-case
- V1
- [KC] [WB] [BP]
[ ] User can develop, test and debug version on the cloud environment for a specific blueprint
- V1
- [KD] [KC] [BP] [WB]
Requires to define the proper python environment : package, configuration, etc.)
Requires a set of tools to facilite the manipulation of the artefact/files[ ] User will rely on dependancy and computation graph integrated management to regenarate elements impacted by any change and evolution
- V1
- [KC]
relies on Parquet file format Storage capacity
./klarity folder in UC repository root to control KC and repo content[ ] business_expert can write down the component description and every thing he is the only one to have the knowledge about/ ability to formalize, without having to use a CLI/complex file format.
- V1
- [KD]
V1 : business artefact creation can be carried out by business roles directly
[ ] Developper have access to an easy to use version of KD
- V1
- [KD]
- To be clarified/specified
[ ] User can work on KD while being offline
- V1
- [KD] [KC] [KS]
WPA application ?
[ ] User can access its data on a Cloud instance of KD while offline.
- V1
- [KD] [KS]
WPA application ?
Cache management must be defined.
Application to Anomaly Detection User Journey
[ ] <customer_other> can easily import Blueprint specified data inside Klarity
- MVP V2 > V1 ?
- [BP] [KD] [KS] [KC]
[ ] User can provide information through interactive artefact that are expecting information.
- MVP V2 > V1 ?
- [KD]
an artefact should provide question/form/elements of input to the user.
- the user will answer the artefact content and push its answer as proposal
- once the proposal validated, the content will be considered as specification for the AI component dev
Cursor to compute ROI interactively and allow the user to make his choice between multiple propositions.
[ ] <customer_other> can carry out data annotation and quality characterization according to blueprint
- MVP V2 > V1 ?
- [BP] [KS] [WB]
[ ] <tech_user> can display handcrafted artefact easily
- MVP V2
- [WB]
[ ] <tech_user> can build the required dataset as specified in associated Blueprint
- MVP V2
- [BP] [WB] [KC]
[ ] <tech_user> can integrate AI component baseline as specified in Blueprint
- MVP V2
- [BP] [WB]
[ ] <tech_user> can start metric computation from models/dataset/feedback/spec as specified in Blueprint
- MVP V2
- [BP] [KS] [WB]
[ ] <customer_other> can operate AI Component on a simulated UC for test purpose, as specified in Blueprint
- MVP V2
- [BP] [KS] [WB]
[ ] <tech_user> can push AI Component simulation metrics and artefact which allow its computatio by WB before being pushed to Karity Dashboard, as required in Blueprint
- MVP V2
- [BP] [KD] [KS] [KC] [WB]
- MVP V2
- [ ] <customer_other> can deploy and operate AI Component in production as specified in blueprint
- V1
- [BP] [KS] [WB]
- [ ] <customer_other> can deploy and operate AI Component in production as specified in blueprint
Storage
[ ] Parquet File format will be used to manipulate dataset meta data and low dimension data
- MVP V2
- [KS]
[ ] User ability to RW data is RBAC
- V1
- [KD] [KS]
User-Interface and Navigation
[ ] User can check the expected action for each artefact group
- MVP V2.0
- [KD]
Navigation overhaul proposal.
By hovering the artefact group or the AI component name (project in the previous terminology), the user will see the expected action (from him / from users / TBD)
[ ] User can only see the active/relevant version
- MVP V2.0
- [KD]
Navigation overhaul proposal.
Depending on his role, user may only see active version / the operation version.
[ ] User can display an AG/A in 4 modes : minimized, normal, full window, full screen.
- V1
- [KD]
Navigation overhaul proposal.
Might be required to progressively switch from MVP V1 mode to this mode. Transition to be defined.
[ ] User can share an AG/A
- V1
- [KD]
Navigation overhaul proposal.
share a link to the AG/A
share/download a .pdf file containing the AG.
[ ] User can compare multiple versions of the same artefact.
- V1
- [KD]
Navigation overhaul proposal.
it is possible to compare an artefact with other versions of it. Using a Diff mecanism. The display will be different depending on the artefact type.
UI mecanism to be refined without the Explorer being implemented.
[ ] User can switch company view between a condensed view and an extended view.
- V1
- [KD]
Navigation overhaul proposal.
Artefact is the entry point. the new company view emphasis on artefact group which can be displayed on the same line (condensed) or on multiple lines (on line per available stage) to compare AG one to one.
[ ] User can check the status of each AI Component deployment project.
- V1
- [KD]
Navigation overhaul proposal.
An AI Component deployment project designate a "project" in the previous terminology. We want the user to be able to quickly see the state of a project (on going, awaiting approval, rejected, ...)
[ ] User can get information regarding the planning/schedule of the AI component deployment project directly in company view.
- V2
- [KD]
Navigation overhaul proposal.
Add a column in company view and define a render view for planning/gantt.
[ ] <business_expert> can navigate between multiple instances of an AI component
- V2
- [KD]
Navigation overhaul proposal.
An AI component can be instanciated multiple times, the business expert need to be able to see the synthesis of all these instances and to see the KPI of each of them. Display groups of AI Components (AI Component = Project in the previous terminology. A line in the company view designates an AI Component)
[ ] User can navigate directly throught the artefact groups and the artefacts without being expected to understand PCIV and versions
- Target
- [KD]
Navigation overhaul proposal.
Artefact is the entry point. The Explorer must allow the user to navigate between artefact groups and artefact (through the blueprint)
[ ] <business_expert> can produce a report
- Target
- [KD]
Navigation overhauld proposal.
It should be possible to download a report (content being pre-configured by blueprint or customized)
[ ] user can bookmark elements
- Target
- [KD]
Navigation overhaul proposal.
bookmarkable elements to be defined (artefact/ artefact group/ ai component/ ... ?)
A tab in the Explorer would store all the bookmarked elements.
[ ] user can select the diff in the Explorer
- Target
- [KD]
Navigation overhaul proposal.
Explorer contains options to select the artefact to be compared
[ ] user can hide/show AG/A in the explorer
- Target
- [KD]
Navigation overhaul proposal.
Explorer contains options to hide/show AG/A
it is also possible, in a single click, to show all previously hidden AG/A
[ ] user can synchronize Explorer with the other navigation frames
- Target
- [KD]
Navigation overhaul proposal.
While navigating the Explorer, the other frames of KD will automatically update. User can deactivate this option.
[ ] user can see all conversations and activities related to the AI components he work (wether the AI component is currently selected or not)
- Target
- [KD]
Navigation overhaul proposal.
The Activity window display, by default, all activities of all AI components/project the user is associated to.
[ ] user can filter the Activity window
- Target
- [KD]
Navigation overhaul proposal.
The Activity window content is subject to different filters :
> Type (All, User related, Comments, Automatic means it is assocatied to user navigation)
> Elements (anything from any Component/project, depending on the version, depending on the AG (checkboxes))
[ ] user can see information for each activity in the Activity window
- Target
- [KD]
Navigation overhaul proposal.
name of the author, type of action, on which A/AG
on which version
an extract of the comment/rational/proposal/review the possibility to extend the extract directly in the Activity window- a link to jump to the associated Artefact
- the day (with time on mouse hover)
[ ] user can minimize the Activity window
- Target
- [KD]
Navigation overhaul proposal.
a button allow to minimize or open the Activity window.
[ ] When there is only one artefact in a group, the group view page should be by passed
- MVP V2
- [KD]
in MVP V1, if there is an artefact group with only one artefact inside, the journey from company is : click on project > click on artefact group (where you already see the artefact content) > it display only the artefact click again on the artefact > it display only the arefact again but now you can interact with it.
[ ] Clicking on an artefact in project view or from the artefact group view, should jump directly to the InDepth view of this artefact.
- MVP V2
- [KD]
in MVP V1, in project view, you can already see the artefact group and the different artefact inside. You cannot click the artefact, you have to click the artefact group name to go to the Artefact Group View, where you will see exactly the same things regarding the artefact group (just, the other artefact group have disappeared) then you must click on a the artefact you want to go on the InDepth view. It is terrible.
[ ] When InDepth view, the size of the artefact window should fit the whole screen.
- MVP V2
- [KD]
[ ] It should be possible to move tab.
- MVP V2
- [KD]
[ ] UX/UI find an alternative to the Not Found message when switching from one PCIV to another one and the requested artefact is not available on the PCIV your going to.
- MVP V2
- [KD]
maybe do not allow to switch to another PCIV when the artefact is not available (display a message explaing this artefact is not available, clicking again will redirect to the PCIV project view)
[ ] User role should be displayed on mouse over
- MVP V2
- [KD]
Yet a user may have multiple role > either have a main role to be displayed or display the most relevant role regarding the current artefact/comment section
[ ] Navigation between conversations of different abstraction levels must be reworked
- MVP V2 > V2 ?
- [KD]
it must goes with the navigation rework. you can't have conversation everywhere and make sure that people speak of a specific thing in the proper space otherwise there will be missing conversation and misunderstanding. Reminder : people are not disciplined and they will say whatever they are thinking about anywhere. This require a deep UX study Display conversattion status : new, available, nothing new, etc.) from project view directly.
[ ] Logout popup is too sensible, fix the timeout.
- MVP V2
- [KD]
[ ] Empty tab must be removed
- MVP V2
- [KD]
[ ] User may quickly see the PCIV tree
- V1
- [KD]
Currently, KD is a maze you spend too much time clicking everywhere to go from somewhere to somewhere, and if you are not the one who created the whole PCIV i think you will quickly get totally lost. We must simplify this navigation.
Maybe a rework of the way version are being displayed in company view, and top-line of project view
[ ] User must be able to display multiple artefact payload inside one single artefact display
- V1
- [KD] [KC]
You have an artefact which is an histogram, and it is relevant to display this histogram with different other artefact. In MVP V1 you have to instanciate this histogram payload as many time as needed, while you'd only want to share it with multiple artefact display.
[ ] User mays be able to display artefact which requires an action for his (user) role.
- MVP V2 / V1
- [KD]
Button shall reorder the artefact (last comments / proposal on top) / or other sorting to be discussed
[ ] User shall be able to generate a report (.PDF) from a validated version.
- MVP V2 / V1
- [KD]
[ ] User shall be able to see the status (work in progress, planeed delivery time, ...) and request for new version.
- V1
- [KD]
The status infomration is computed from available metrics vs required metrics to procude expected artefacts for the currently expected version.
[ ] User shall be able to list/filter/sort artefacts display in project and indepth views
- V1
- [KD]
[ ] As a complement to "User may quickly see the PCIV tree" a dedicated view to see the version tree of a project with all décisions action on PCIV level.
- V1
- [KD]
Artefacts
[ ] Project ROI
- MVP V2
- [KD]
ROI artefact must display the following information
The estimated ROI
The real ROI in time
The values and formulas used to compute ROI The possibility to esimate different ROI with different values and to compare with the estimated ROI and the real ROI.[ ] Comparison & Drift artefact group
- MVP V2
- [KD]
The Comparison & Drift artefact group will contain at least two artefact
[ ] It should be possible to instanciate artefact directly in the artefact list type (like for the sample view)
- MVP V2
- [KD] [KC]
each artefact must be declared independantly from KC. Yet, it would be useful to be able to instanciate one directly in the artefact list when describing the list element.
technical improvement needed
- [LCA] Rework of backend communication, project fetch, data format.
- V1
Split the configuration between HMI confirugation, artefacts / metrics links constraint.
- [KD]
To be Integrated
To Be Integrated capacities have been discussed and are validated for integration. This is the Backlog that can be used to push dev activities to the dev teams.
User-Interface and Navigation
[ ] User can use a rich text edition mode to write comments, on KD.
- MVP V2
- [KD]
[ ] Tooltip must have a timeout after first "mouse over"
- MVP V2
- [KD]
when cursor is over an artefact in KD, a tooltip is displayed, this tooltip never vanish, which is a problem.
[ ] BUG : refreshing the page you are on will display an error message, leaving Klarity Dashboard : "An error occurred". This is a big user experience problem.
- MVP V2
- [KD]
user should be able to share url !
[ ] BUG : HTML Content type cannot be scrolled once it is validated.
- MVP V2
- [KD]
[ ] BUG : deleting a project "toto" will remove every project with the same prefix : toto2, toto_projet, toto_test, ... will also be removed.
- MVP V2
- [KD]
[ ] BUG : an artefact group containing an html_content and other type of artefact will generate an error 500 when trying to open project.
- MVP V2
- [KD]
User-Journey
[ ] User can register/create an account on KD
- MVP V2 > V1 ?
- [KD]
[ ] User can create a dataset, generate associated metrics and push the corresponding artefact on KD
- MVP V2 > v1 ?
- [KD] [KS] [KC] [WB]
Dev n Debug
[ ] It is possible to remove a configuration from KD
- MVP V2
- [KD] [KC]
[ ] it is possible to see to which configuration (name and version) a PCIV is associated to, on KD
- MVP V2
- [KD] [KC]
Integrated But Not Validated
Integrated But Not Validated capacities have been declared as integrated by the dev team. This integration and the accordance with the expected capacity must now be validated.
User-Interface and Navigation
- [ ] User can request a new scoping or a new development PCIV from the comment section
- MVP V1.1
- [KD]
Not validated because it generates an issue to gitlab and the connection has not beed properly configured to do so.
Artefact and metric display
- [ ] User-can display a complex type of metric
- MVP V1.1
- [KD]
See if the roc_curve type, which use echart, can do the job. altought it is required to characterize a "complex metric" first.
Extension system
- [ ] Extension system must be operational (see issue 23)
- MVP V2
- [KD]
it must be possible for a newcomer to dev and implement its own extension
Integrated and Validated
Integrated and Validated capacities have been validated as being integrated and operational.
User-Journey
Visual Inspection Anomaly Detection Use-Case Journey
Generic User-Journey Capacities
- [x] User can push its and made it available through a cloud-based instance of KD.
- MVP V1
- [KD] [KS] [KC]
- [x] Administrator can create users on KD and give specific right and access
- MVP V1
- [KD]
- [x] User can push its configuration and payload to KD
- MVP V1
- [KD] [KC]
- [x] User can navigate Scoping PCIV, comment, propose validation/rejection (with a rationale), review proposals and validate/reject artefacts or the whole PCIV directly, on KD
- MVP V1
- [KD]
- [x] User can push a new PCIV on KD
- MVP V1
- [KD] [KC]
- [x] User can push a new configuration on KD
- MVP V1
- [KD] [KC]
- [x] User can push multiple version of the same configuration on KD
- MVP V1
- [KD] [KC]
- [x] User can create multiple project for the same configuration or for multiple version of the same configuration and see them on KD
- MVP V1
- [KD] [KC]
User-Interface and Navigation
- [x] User-journey through KD must be complete and all GUI element must be integrated or easy to integrate/update (no major modification).
- MVP V1
- [KD]
- [x] It is possible to use display/hide the log bar of KD to display the logging information.
- MVP V1
- [KD]
- [x] It is possible to display information on the KD status bar for extension, and to see the online/diff status of the current session.
- MVP V1
- [KD]
- [x] User can log on KD.
- MVP V1
- [KD]
- [x] User can edit his KD profile settings.
- MVP V1
- [KD]
- [x] User can edit/add/remove a project members (and roles) on KD.
- MVP V1
- [KD]
- [x] User can see the project on the KD Company view.
- MVP V1
- [KD]
- [x] User can see a project 'Project view" on KD
- MVP V1
- [KD]
- [x] User can see an artefact group, view, on KD
- MVP V1
- [KD]
- [x] User can see an artefact InDepth, view, on KD
- MVP V1
- [KD]
- [x] User can read other user comments at any abstraction level, on KD
- MVP V1
- [KD]
- [x] User can write comments at any abstraction level, on KD
- MVP V1
- [KD]
- [x] User can propose a decision at any abstraction level, on KD
- MVP V1
- [KD]
- [x] User can review a decision proposal at any abstraction level, on KD
- MVP V1
- [KD]
- [x] User can validate or reject a proposal at any abstraction level, on KD
- MVP V1
- [KD]
- [x] User can for a validation or rejection decision at any abstraction level, on KD
- MVP V1
- [KD]
- [x] It is possible for different user to interact on the same project, on KD
- MVP V1
- [KD]
Artefact and metric display
- [x] User can display an artefact_list (requirement_list / requirement_coverage / sample_list) type on each abstraction level, on KD
- MVP V1
- [KD]
- [x] User can display a confusion_matrix artefact type on each abstraction level, on KD
- MVP V1
- [KD]
- [x] User can display an html_content artefact type on each abstraction level, on KD
- MVP V1
- [KD]
- [x] User can display an image_overlay artefact type on each abstraction level, on KD
- MVP V1
- [KD]
- [x] User can display a roc_curve (and any kind of echart curve) artefact type on each abstraction level, on KD
- MVP V1
- [KD]
Rename this artefact type into chart type as it uses echart and can display any kind of echart with some customization
- [x] User can display a value_ration artefact type on each abstraction level, on KD
- MVP V1
- [KD]
Maybe remove this artefact type as the roc_curve type can do it with a better render.
Deployment
- [x] It is possible to locally deploy Klarity
- MVP V1
- [KD]
- [x] It is possible to deploy different instances of Klarity on cloud
- MVP V1
- [KD] [KS]