QA Web
Feb 05, 2020

Netgear-Galaxy | QA for seamless product performance

Netgear builds networking products that connect people, and power businesses with wireless internet connectivity, streaming and storage solutions.

Being the industry leading enterprise, Netgear is always innovating and looking to scale. Their reliable range of devices and solutions has made them a household name. They have numerous devices across different verticals such as Home security, SMB, Enterprise, and Retail. All of their devices come with a companion app from which the devices are managed.

We came onboard to provide support for firmware upgrades, security patches and similar updates for the devices while improving the onboarding and user flow of the companion apps.

The Challenge

When testing something like a website, it’s easier to run through the scenarios that most users will encounter as they scroll through their laptops, smartphones or tablets. They realized it’s difficult to replicate a users’ environment with wi-fi routers, IoT and other surveillance devices.

It was also crucial to ensure that the devices are performing as expected individually and in a mesh network after every security patch fix or firmware upgrade.

We integrated into their team to solve two major pain points:

1. UX

Netgear encountered increasing drop offs during the release cycle of their products. UX issues can’t be addressed with pure intuition so we turned to our Analytics team for insights on user behaviour. Data helped us pinpoint the screens where users were dropping off the most because they couldn’t understand the feature’s value proposition.

2. Scant Resources

Due to lack of testing resources and expertise, they were unable to perform 100% testing on their myriad of products. They wanted to ensure that every possible combination of inputs and every possible sequence of device states are tested, and all outputs are verified for every case.

For that they needed testing experts as an extension to their team who can give them the low-down about all the discrepancies in the functionalities covered.

We provided Netgear a comprehensive range of testing solutions for conformance, functional and performance testing of devices and networks. We also provided advice around the efficient resources to be adopted, the delivery process, and the timeline needed to ensure the expected quality level.

The Goal

Improve development and testing processes to prevent defects during the product development lifecycle, and ultimately improve user experience.

Considering Netgear’s business needs, the following QA testing process was adopted:

  • Proposed automated testing environment to test large numbers of devices
  • Perform Visual QA testing to validate visual correctness of an application.
  • Test the functional UI/UX of the device network management flow.
  • To check non-functional aspects (performance, usability, reliability, etc.) of the software application.
  • Test multiple device configurations to ensure application compatibility and responsiveness.

Visual QA

During the designing phase we introduced a visual QA to ensure a pixel perfect implementation of the designs through and through. We compared the screenshots with the initial ‘baseline’ images, and reported the differences.

In this way, we performed full-page validation to get more coverage that led us to catch unexpected bugs. This approach helped Netgear’s developers understand how visual element components will render across various systems, and find the root cause of all the visual errors encountered.

UX testing

While verifying the bugs submitted by Netgear’s team, we found some unaddressed UX issues because of improper priority definitions. We suggested redefining our priorities and addressed the UX issues first.

The onboarding screen in one of their apps had two CTAs, one saying ‘Done’ and the other saying ‘Next’. Visibility of ‘Done’ CTA at the wrong point in the user flow prompted users to quit the onboarding abruptly by pressing ‘Done’ without even completing the onboarding.

This helped Netgear team rethink their user flow and onboarding and eventually resolve the drop-off issues.

Functional Testing

There is at least an app for every single one of the Netgear devices even if it’s their digital canvas Meural. We had to test login cases, connection cases, and gesture UI cases for the relevant devices. We tested each part of the system against functional specifications/ requirements, seeking answers to various questions like:

  • Are the users able to login to a system after entering correct credentials
  • If incorrect credentials are entered, is the system informing the user and reloading the login page
  • Whether the actual state of the application is the expected one

Compatibility Testing: We addressed the compatibility issues that arise with the connected devices to understand how various devices interact with the digital environment. There were several validation considerations such as device compatibility with different versions, screen resolution requirements on different devices started from the device layer to the network layer that were executed in this type of testing.

We checked the compatibility of device versions and protocols through tracking connection delay, assessment of data integrity, safety evaluation and scalability testing.

The testing was carried out on different combinations of OS/browser/resolutions and on different iOS and Android devices.

Documentation

We used Jira to manage the entirety of the testing cycle from test planning to design, execution and test reporting. Upon the project completion, we provided with the following testing artifacts:

The Quality Report with the description of all defects by their priority, devices, and functional modules of the devices and apps where they were detected.

List of the conducted functional tests.

  • Structured recommendations on the system environment and device quality improvement.
  • Verified the story bugs their team provided while reporting new bugs
  • We on average addressed around 200-400 bugs pertaining to 10-12 releases per month for different devices.

Results

With the inclusion of Galaxy, Netgear sped up their feedback cycle, subsequently resulting in faster time to market with a polished product. More importantly we helped them with a substantial reduction in drop-offs. The impacts were visible via user-satisfaction being reflected in the user reviews on Android and Apple marketplaces.

  • Reduced testing time of different devices per execution
  • Faster Time-to-Market
  • Better onboarding with reduced drop-off rate
  • Faster bug recognition and improved UI
  • Better overall app quality and user experience

By Aaditya Mandloi

About Galaxy Weblinks

We specialize in human-centric user experience design services to our clients across the globe. Our innovative ideas, creative designs, industry best practices, and processes help us improve user satisfaction and solve complex design problems. Contact us for a free consultation!

Connect with us

Connect with us

Are you planning to build a software?

Connect with us

Which delivery are you looking for?

Connect with us

Which delivery are you looking for?

Connect with us

What is your budget range?