Spatial Module Usability
Evaluating how system implementers perceive and interact with the "spatial module" in the Honeywell Forge Onboarding Portal to identify usability and workflow gaps.
Product
Honeywell, Onboarding Portal
Platform
Web app
TL;DR
Overview
To better understand how system implementers interact with the Honeywell Forge Onboarding Portal, I conducted a usability study focused on its "Spatial module"—specifically, the process of bulk-creating spatial instances.
This research study involved administering the System Usability Scale (SUS) and analyzing task performance to measure efficiency and effectiveness.
Role
UX research lead
Completion
January 2025
Problem statement
System implementers using the Onboarding Portal rely on the "spatial module" to bulk-create and configure instances of spaces (I.E., floors and offices). But the usability of this workflow has not been formally evaluated—making it unclear how intuitive and efficient the experience is. Without baseline usability metrics, it's difficult to identify improvement opportunities in the Onboarding Portal.
To inform opportunities for the tool's experience, a usability and SUS evaluation is needed to surface pain points and establish baseline metrics.
Goals
Assess the current SUS (system usability scale) for the Onboarding Portal’s Spatial module
Measure average time-on-task for bulk-creating spatial instances
Identify the task error and success rates to uncover points of friction in the workflow
Approach
Methodology
60-minute, one-on-one interviews to conduct three activities with each participant in the demo environment for the Honeywell Forge Onboarding Portal:
Review a CSV template for “Floor” (an object in the Portal)
Create instances of floors using the given CSV template
Modify one instance of a floor
Participants
I interviewed 4 participants from Honeywell's internal users:
3 of 4 work in the Industrial Automation business unit
1 of 4 works in the Building Automation business unit
Participant job titles:
2 of 4 participants are SR Advanced Application Engineers
1 of 4 participants is a Lead Systems Engineer
1 of 4 participants is a Software Engineer II
Stimuli
I. CSV template for a "Floor"
Participants were given access to a pre-filled out CSV template for “Floors” downloaded from the Onboarding Portal beforehand.
I sent this template to the participant once we started the interview, and we reviewed it together.
II. Honeywell Forge Onboarding Portal demo
Participants were given a "test" login to access to a demo environment for the Onboarding Portal to complete the remaining tasks before reviewing the SUS evaluation form.
Findings
Landing page doesn’t communicate functionality or value
Import feature is difficult to discover
Import feature lacks sufficient system feedback
Engineers have various mental models of "core objects"
Landing page doesn’t communicate functionality or value
All participants described the Onboarding Portal as a system to manage password permissions or set up user profiles for their own account or their customer’s account upon first impression (which is not correct).
“This looks like a profile setup or user profile setup, and I would say organization settings.”
-P4, Lead Systems Engineer, Building Automation
Import feature is difficult to discover
2 of 4 participants failed to find the import feature while creating spatial instances from the provided CSV template (Task 1) – requiring assistance to proceed with the task.
“I am missing something… where is that link to do the import should be. This is a moment where I can use your help.”
–P2, Sr. Advanced Application Engineer, Industrial Automation
Import feature lacks sufficient system feedback
3 of 4 participants were unsure if the CSV file uploaded in the import table correctly (task 1) and expected more feedback from the system (I.E., status indicators or error messages).
“So, it uploaded, I’m not sure it worked… I’m not really sure what I’m looking at here.”
-P3, Sr. Advanced Application Engineer, Industrial Automation
Engineers have various mental models of "core objects"
All participants showed confusion when navigating the core objects (I.E., Assets and Spaces) while using the Onboarding Portal.
“I’m going to start with the assets menu… my memory came back, and I remember that there’s a difference between spatial assets and physical assets.”
-P1, Software Engineer II, Industrial Automation
Key insights
Participants with greater familiarity of the Onboarding Portal gave higher SUS scores
The import feature feels disconnected from the "object"
Object creation forms and CSV template’s headers lack clarity
Participants with greater familiarity of the Onboarding Portal gave higher SUS scores
P1 and P2 had the greatest familiarity with the Onboarding Portal to configure customers and gave the highest SUS score (both totaling an 80).
* Participants asked to rate familiarity with the Onboarding Portal & CVS template usage from 1-Very unfamiliar to 5-Very familiar)
** DNC (Did not complete) – Both participants needed assistance in finding the import feature of the Onboarding Portal
The import feature feels disconnected from the "object"
Current import feature doesn’t indicate that it’s for the specific object model and feels like a different experience compared to the object’s detail page (“Spaces” data table).
Note: The import page has a different header, data table layout, and uses non-Forge UI Design System components and patterns.
“The import felt like I was taken to a whole different functional block when I went to import that CSV.”
-P3, Sr. Advanced Application Engineer, Industrial Automation
Object creation forms and CSV template’s headers lack clarity
More documentation or help is needed to fill out the form to manually create an instance/ template of an object as well as the fill out CSV templates correctly.
I.E., add constraints vs. free-form input or add help text.
“There is some obfuscation between what I see on the screen (Portal) and what is up here in the headers (CSV).”
– P1, Software Engineer II, Industrial Automation
“Whether it’s documented in documentation or some kind of help… might be helpful on what am I actually filling in here?"
-P3, Sr. Advanced Application Engineer, Industrial Automation
Conclusion
Outcome
This evaluation revealed several areas where the "spatial module" could better support system implementers, such as the landing page experience and general information architecture of the Onboarding Portal.
This research study provided actionable insights to enhance the onboarding experience, but also equipped our team with baseline SUS scores to compare future scores and measure improvement.
Next steps
As a result of the research insights, future opportunities were identified to:
Redesign the landing page to highlight functionality and guide users
Improve the visibility and affordances of the "Import" feature
Assess the information architecture to ensure it aligns terminology with users’ mental models