Back to all articles
GooglePlayClosedTesting
AndroidAppTesting
GooglePlayConsole

Google Play Closed Testing: How to Handle Testers Using Different Android Devices

Saksham Goyal

Saksham Goyal

Author

October 30, 2026

Published

47

Views

Google Play Closed Testing: How to Handle Testers Using Different Android Devices

Image source: Unsplash · Google Play Closed Testing: How to Handle Testers Using Different Android Devices

Learn how to manage Google Play Closed Testing across different Android devices, versions, screen sizes, networks, and performance levels for better app testing.

One tester has a Samsung.

Another uses a Pixel.

Someone has a OnePlus.

Another has an older Xiaomi.

And then there is the person testing your app on a tablet that you completely forgot existed.

Welcome to Android app testing.

Device diversity is one of the biggest advantages of a real-user Closed Test, but it can also make testing harder to organize. Different Android versions, screen sizes, manufacturers, hardware capabilities, network conditions, and device performance can expose problems that never appear on the developer's own phone.

Google recommends using Closed Testing to gather targeted feedback from a wider set of testers before production. Android's developer guidance also warns that testing only on a single device can allow problems on other screen sizes and form factors to go unnoticed.

The solution is not to test every Android device on Earth. That would be a wonderfully terrible use of everyone's time.

Instead, build a balanced device-testing strategy.

Why Different Android Devices Matter

An app can behave perfectly on one phone and still have problems elsewhere.

For example:

  • A button may be cut off on a smaller display.

  • A layout may look strange on a large screen.

  • A feature may behave differently on another Android version.

  • A slower device may expose performance problems.

  • A manufacturer-specific implementation may affect notifications or background behavior.

  • A weak network connection may expose loading problems.

  • Different screen densities can affect UI rendering.

Android's guidance specifically notes that problems can go unnoticed when development relies on a single device and recommends testing across different screen and window sizes.

That is exactly why having testers with different devices can be valuable.

1. Don't Give Every Tester the Same Phone

If all 12 of your testers use the same phone model, you may get 12 opinions about one device environment.

That is useful, but limited.

A more diverse group might look like:

Tester

Brand

Device Type

Android

Main Focus

T01

Google

Pixel

Recent

Core features

T02

Samsung

Galaxy

Recent

UI + navigation

T03

OnePlus

Phone

Recent

Performance

T04

Xiaomi

Redmi

Recent

Compatibility

T05

Motorola

Phone

Older

Performance

T06

Samsung

Galaxy

Older

Android compatibility

T07

Google

Pixel

Recent

Notifications

T08

Xiaomi

Redmi

Recent

Network behavior

T09

OnePlus

Phone

Recent

Login

T10

Samsung

Large display

Recent

UI

T11

Android tablet

Tablet

Recent

Large screen

T12

Other

Phone

Recent

General testing

You do not need exactly this combination.

The principle is what matters:

Different testers should expose different environments whenever practical.

2. Track Device Information

Don't simply record:

Rahul is testing.

Record:

Rahul → Samsung Galaxy S23 → Android 15

Your tester tracking sheet can include:

  • Tester ID

  • Google Account

  • Device manufacturer

  • Device model

  • Android version

  • Screen size

  • Testing focus

  • Opt-in date

  • Feedback

  • Current status

For example:

Tester

Device

Android

Focus

Status

T01

Pixel 8

15

Core features

Active

T02

Galaxy S23

14

UI

Active

T03

OnePlus 12

15

Performance

Active

T04

Redmi Note

14

Compatibility

Active

This makes bug investigation much easier.

If three testers report the same issue and all use similar devices, you have an important clue.

3. Test Multiple Android Versions

Device diversity isn't only about manufacturers.

Android versions matter too.

A feature can behave differently across Android versions because of differences in APIs, permissions, system behavior, or platform restrictions.

Try to understand which Android versions your target audience actually uses.

Then make sure your tester group provides useful coverage across those versions.

For example:

Recent Android version

Test:

  • Latest platform behavior

  • New permission behavior

  • Modern UI

  • Recent system restrictions

Older supported Android version

Test:

  • Compatibility

  • Performance

  • Permissions

  • Layout

  • Core functionality

Do not deliberately test unsupported Android versions just to make your spreadsheet look impressive.

Test the versions your app is actually intended to support.

4. Test Different Screen Sizes

Screen size is one of the easiest ways for UI problems to hide.

A screen that looks perfect on your development phone might look completely different on:

  • Small phones

  • Large phones

  • Foldables

  • Tablets

  • Landscape layouts

  • Split-screen environments

Android's official guidance recommends testing different screen and window sizes because single-device development can allow layout problems to go unnoticed.

Ask testers to check:

  • Text wrapping

  • Buttons

  • Images

  • Cards

  • Navigation bars

  • Dialog boxes

  • Forms

  • Keyboard behavior

  • Scrolling

  • Orientation changes

A particularly useful instruction is:

“If anything looks cut off, overlapping, unusually small, or difficult to tap, take a screenshot and report it.”

Screenshots turn vague complaints into actionable bugs.

5. Test Both Fast and Slow Devices

Not everyone owns a flagship phone.

Your app may feel incredibly fast on a high-end device but frustratingly slow on a lower-end phone.

Ask testers to observe:

  • Startup time

  • Screen transitions

  • Loading indicators

  • Scrolling

  • Image loading

  • Search

  • Login

  • Background operations

  • Memory-related problems

  • Battery impact

You don't need to benchmark every phone scientifically.

You need to discover meaningful differences in real-world use.

6. Test Different Network Conditions

Your app doesn't live in a perfect Wi-Fi laboratory.

Testers may use:

  • Fast Wi-Fi

  • Slow Wi-Fi

  • Mobile data

  • Weak signal

  • Intermittent connections

  • Temporary loss of connectivity

Ask testers to try important workflows under different network conditions.

For example:

Login

What happens if the connection disappears during login?

Upload

What happens if the network becomes slow?

Payment

What happens if the connection drops during a transaction?

Content loading

Does the app show a useful loading state or simply freeze?

These scenarios can reveal serious usability problems.

7. Give Testers Device-Specific Tasks

Don't ask every tester to do everything.

Assign specific areas.

Group A: Different Android Versions

Focus on:

  • Permissions

  • Notifications

  • System behavior

  • Compatibility

Group B: Different Screen Sizes

Focus on:

  • Layout

  • Text

  • Navigation

  • Orientation

  • Forms

Group C: Lower-End Devices

Focus on:

  • Startup

  • Performance

  • Scrolling

  • Memory

  • Loading

Group D: Different Manufacturers

Focus on:

  • Notifications

  • Background behavior

  • Permissions

  • Device-specific issues

This creates broader coverage without asking every tester to repeat the same work.

8. Ask Testers to Report Their Device

When a tester reports:

“The app crashes.”

You need more information.

Ask them to provide:

Device: Samsung Galaxy A54
Android: Android 14
App version: 1.0.7
Feature: Checkout
Problem: App closes after tapping Pay
Steps:

  1. Open cart

  2. Select payment

  3. Tap Pay

  4. App closes

Now the development team has something useful.

A bug report without environment information is basically a mystery novel nobody asked for.

9. Keep the App Version in Your Bug Reports

Device information alone isn't enough.

Suppose:

  • Tester A reports a crash on version 1.0.6.

  • You release version 1.0.7.

  • Tester B reports the same issue.

Is it still happening?

You need to know which build they are using.

Record:

  • App version

  • Version code

  • Device

  • Android version

  • Feature

  • Steps to reproduce

Google Play distributes releases through testing tracks, and testers receive updates to the test version as new releases become available.

That makes version tracking important throughout the test.

10. Don't Assume a Device Problem Is an Android Problem

Suppose one tester reports:

“Notifications aren't working.”

Don't immediately conclude:

“Android notifications are broken.”

Investigate.

The problem could be:

  • App notification permission

  • Device settings

  • Battery optimization

  • App configuration

  • Network issue

  • Incorrect implementation

  • Manufacturer-specific behavior

Test the same feature on another device.

If several devices show the same issue, it is more likely to be an application-level problem.

If only one device or manufacturer shows it, investigate device-specific behavior.

11. Use Screenshots and Screen Recordings

Visual problems are much easier to diagnose when you can see them.

Ask testers to attach:

  • Screenshot

  • Screen recording

  • Error message

  • Relevant device information

For example:

Bug: Button overlaps text
Device: Redmi Note 13
Android: 14
App: 1.0.4
Screenshot: Attached

This gives developers immediate context.

12. Test the App in Portrait and Landscape

If your app supports orientation changes, test both.

Check:

  • Login

  • Forms

  • Video

  • Navigation

  • Dialogs

  • Lists

  • Images

  • Keyboard interaction

A screen that works perfectly in portrait can become unusable in landscape.

If your app intentionally supports only one orientation, make sure that behavior is deliberate and consistent.

13. Test Tablets and Foldables When Relevant

If your app is intended to work on larger screens, include larger-screen testing.

Android's testing guidance specifically covers different screen and window sizes and recommends testing across different form factors.

You don't necessarily need to buy every device.

For additional technical coverage, Android Developers points to services such as Firebase Test Lab for testing on specific real devices and emulators, including devices you may not physically own.

Real-user testing and automated/device-farm testing can complement each other.

14. Use Your Closed Test for Real-World Feedback

The purpose of Closed Testing is not simply to prove that the APK installs.

Google describes closed testing as a way to test pre-release versions with a selected group and gather targeted feedback before broader availability.

So ask testers to use the app naturally.

For example, if you're testing a shopping app, don't only ask them to open every screen.

Ask them to:

  1. Search for a product.

  2. Open product details.

  3. Add something to a cart.

  4. Modify the cart.

  5. Complete the intended workflow.

  6. Report anything confusing.

Realistic usage can reveal problems that scripted button-tapping misses.

15. Don't Confuse Device Diversity With Tester Quantity

Having 20 testers does not automatically mean you have good device coverage.

You might have:

20 testers

but:

18 identical Samsung models

That's still a narrow device sample.

Likewise, having 12 different devices does not automatically mean you have enough useful tester engagement.

Google's Production Access process asks developers to describe tester engagement, whether testers used available features, whether usage matched expected production behavior, and what feedback was received.

So think about both:

Tester quantity + Tester diversity + Tester engagement

16. Build a Device Coverage Matrix

A simple matrix can make your testing strategy much clearer.

Area

Device A

Device B

Device C

Device D

Login

✅

✅

✅

✅

Signup

✅

✅

—

✅

Payments

✅

—

✅

✅

Notifications

✅

✅

—

✅

Dark Mode

✅

✅

✅

—

Landscape

—

✅

—

✅

Poor Network

✅

—

✅

—

This lets you see where coverage is missing.

You don't need every tester to test every feature.

You need enough meaningful coverage to identify important problems.

17. Prioritize Important Features

Not every feature deserves the same amount of testing time.

For a typical app, prioritize:

High Priority

  • Signup

  • Login

  • Main user workflow

  • Payments

  • Data submission

  • Account management

  • Notifications

  • Critical API interactions

Medium Priority

  • Search

  • Filters

  • Profile

  • Settings

  • Secondary screens

Lower Priority

  • Rarely used features

  • Cosmetic settings

  • Non-critical animations

Your priorities depend on the app.

A payment app should not spend more testing time on a fancy animation than on transaction reliability. That would be a fascinating way to fail.

18. What to Do When Only One Device Has a Problem

Don't immediately ignore it.

Follow this process:

Step 1

Confirm the problem.

Step 2

Record the exact device and Android version.

Step 3

Ask another tester with the same or similar device to reproduce it.

Step 4

Try the same workflow on another manufacturer.

Step 5

Check whether the issue is reproducible.

Step 6

Investigate the root cause.

Step 7

Fix it if necessary.

Step 8

Release the updated build.

Step 9

Ask testers to retest.

This turns one unusual report into useful diagnostic information.

19. Retest After Fixes

Finding a bug is only half the job.

After fixing it:

Bug reported

↓

Fix implemented

↓

New build released

↓

Tester installs update

↓

Original device retests

↓

Different device retests

↓

Result documented

This is especially useful for device-specific issues.

A fix that works on a Pixel may still fail on a Samsung.

20. Keep Your Tester Spreadsheet Organized

A practical device-testing tracker might look like this:

ID

Tester

Device

Android

Network

Focus

App Version

Status

T01

Rahul

Pixel 8

15

Wi-Fi

Core

1.0.5

Active

T02

Priya

Galaxy S23

14

5G

UI

1.0.5

Active

T03

Amit

Redmi

14

4G

Performance

1.0.5

Active

T04

Neha

OnePlus

15

Wi-Fi

Login

1.0.5

Active

T05

Karan

Galaxy A-series

13

4G

Compatibility

1.0.4

Retest

This becomes extremely useful when reviewing bugs.

21. Use Play Console Alongside Your Own Tracking

Play Console is the official testing environment.

Your spreadsheet is your operational management tool.

Use Play Console for:

  • Testing tracks

  • Tester access

  • Releases

  • Distribution

  • Test feedback

  • Test configuration

Use your tracker for:

  • Device models

  • Android versions

  • Assigned tasks

  • Bug status

  • Test coverage

  • Follow-ups

  • Retesting

Google Play supports closed testers through email lists and Google Groups, and developers can provide a feedback URL or email address for testers.

22. Don't Try to Test Every Android Device

This is important.

There are far too many Android devices to test individually.

Instead, create representative coverage.

Consider:

  • Popular manufacturers among your audience

  • Supported Android versions

  • Low-end and high-end hardware

  • Small and large screens

  • Different network conditions

  • Relevant form factors

If your analytics show that 70% of your expected users have a particular type of device, prioritize that environment.

Testing should be based on risk and real users, not on collecting phone models like Pokémon.

23. When You Have Only 12 Testers

If you're completing the closed test required for Production Access, you may be working with a relatively small group.

In that situation, don't try to assign every possible device configuration.

Instead, distribute your testers strategically.

For example:

12 testers

  • 3 Samsung

  • 2 Pixel

  • 2 Xiaomi

  • 2 OnePlus

  • 1 Motorola

  • 1 other manufacturer

  • 1 tablet or larger-screen device, if relevant

This is only an example.

Your actual distribution should reflect your app's target audience.

The objective is meaningful coverage, not a magic manufacturer ratio.

24. Prepare Testers Before They Start

Give testers clear instructions.

Tell them:

Your device: Please tell us your model and Android version.

Your task: Test login, navigation, and the main workflow.

What to report: Crashes, errors, slow loading, broken layouts, confusing screens.

What to include: Screenshot, device, Android version, app version, and steps to reproduce.

Google recommends clear tester instructions and a clear feedback channel for closed testing.

Good instructions produce better testing.

25. Use a Device Testing Checklist

Before finishing your test, verify:

Device Coverage

Multiple manufacturers tested

Supported Android versions tested

Different screen sizes tested

Relevant tablets/foldables tested

Low-end hardware tested where relevant

High-end hardware tested

Feature Coverage

Signup

Login

Main feature

Navigation

Notifications

Forms

Payments, if applicable

Error handling

Offline/poor-network behavior

Technical Coverage

Startup

Loading

Scrolling

Background behavior

Orientation

Memory/performance

Crashes

Feedback

Device information recorded

Android version recorded

App version recorded

Screenshots collected

Bugs documented

Fixes retested

Example 14-Day Device Testing Plan

Days

Focus

Day 1

Installation and device compatibility

Days 2–3

Signup, login and onboarding

Days 4–5

Core app features

Days 6–7

UI and different screen sizes

Days 8–9

Performance and network conditions

Days 10–11

Notifications and background behavior

Days 12–13

Bug fixes and regression testing

Day 14

Final device-specific checks and feedback

This is a practical framework, not a Google-mandated daily schedule.

The important thing is that your testing should reflect your app and its expected users.

Common Mistakes When Testing Different Android Devices

Mistake 1: Testing Only the Developer's Phone

The classic.

It works perfectly on your phone, therefore obviously the entire Android ecosystem must be fine.

Unfortunately, Android did not sign that agreement.

Mistake 2: Ignoring Older Supported Versions

If your app supports an older Android version, test it.

Mistake 3: Ignoring Screen Sizes

A beautiful UI on one phone can become a cramped disaster elsewhere.

Mistake 4: Not Recording Device Information

Without device details, diagnosing device-specific bugs becomes much harder.

Mistake 5: Treating Every Device as Identical

Manufacturer and hardware differences can matter.

Mistake 6: Testing Only Wi-Fi

Real users use mobile data and unreliable connections.

Mistake 7: Fixing a Bug Without Retesting the Original Device

Always verify the fix where the problem was discovered.

Mistake 8: Collecting Devices Instead of Feedback

Device diversity is useful only when it produces meaningful testing.

When Should You Use a Device Testing Service?

If you don't physically have access to enough devices, automated device testing can supplement your human testers.

Android Developers specifically points to Firebase Test Lab and similar device-farm services for accessing real devices and emulators that you may not own.

This can be particularly useful for:

  • Rare devices

  • Tablets

  • Foldables

  • Different screen sizes

  • Automated regression testing

  • Reproducing device-specific issues

Human testers and automated device testing solve different problems.

Human testers:
Real-world behavior and usability.

Device farms:
Broader technical coverage and repeatable tests.

Using both can give you stronger coverage.

How Different Devices Help With Production Access

For developers subject to Google's testing requirements, the Production Access application asks about tester engagement, feature usage, expected production behavior, feedback, and changes made based on testing.

That means your testing shouldn't simply be:

“We had 12 testers.”

A stronger process is:

“Our testers used the core features across different Android devices and versions. We identified several issues with navigation and performance, fixed them, and asked testers to retest the affected workflows.”

That demonstrates an actual testing process.

Be truthful and describe what actually happened.

Final Device Testing Strategy

A strong Android Closed Test does not require hundreds of phones.

It requires intentional coverage.

Think about:

Different manufacturers

  •  

Different Android versions

  •  

Different screen sizes

  •  

Different performance levels

  •  

Different network conditions

  •  

Realistic user behavior

  •  

Structured feedback

=

Better testing

Google Play's Closed Testing system is designed to help developers gather targeted feedback from selected testers before wider release.

Your goal is not to prove that your app works on one perfect phone.

Your goal is to discover where it doesn't work before your users discover it for you.

Final Checklist

Before completing your Closed Test:

Testers use more than one device model

Multiple manufacturers are represented where practical

Supported Android versions are covered

Different screen sizes are tested

Relevant tablets or foldables are considered

Lower-end devices are included when relevant

Wi-Fi and mobile data are tested

Core features are tested across multiple devices

Device-specific bugs are documented

Screenshots or recordings are collected

App version is recorded for each bug

Fixes are retested

Tester feedback is organized

Device coverage is documented before Production Access

Conclusion

Handling testers on different Android devices is not about forcing every tester to follow the same exact setup.

It is about using their differences to your advantage.

Track their devices and Android versions. Assign testing areas based on your app's risks. Test important workflows across different environments. Pay attention to screen sizes, performance, connectivity, and device-specific behavior. When a bug appears, record enough information to reproduce it and retest the fix.

Google recommends Closed Testing for targeted feedback before broader release, while Android's own testing guidance highlights the importance of testing across different screen sizes and form factors.

The best device-testing strategy is therefore not:

“How many phones can we test?”

It is:

“Which different environments could expose a problem before our users do?”

That is the question worth answering before you hit Production.

FAQs

1.Should all Google Play testers use different Android phones?

No. Google does not require every tester to use a different device. However, having useful device diversity can expose compatibility and usability issues that may not appear on a single device.

2.Which Android devices should I include in testing?

Prioritize devices and Android versions that represent your expected users. Include different manufacturers, supported Android versions, screen sizes, and performance levels where practical.

3.Should I test older Android versions?

Yes, if your app supports them. Test the Android versions that your application is intended to support.

4.Should testers use both Wi-Fi and mobile data?

For apps that depend on network connectivity, it is useful to test both. Poor or interrupted connections can expose loading, timeout, retry, and error-handling problems.

5.Should I test tablets?

If your app supports tablets or your target audience uses them, yes. Android recommends considering different screen and window sizes and form factors during testing.

6.What information should testers provide when reporting a bug?

Ask for the device model, Android version, app version, steps to reproduce, expected behavior, actual behavior, and a screenshot or recording when useful.

7.Do I need to test every Android device?

No. There are too many devices to test individually. Build representative coverage based on your target audience, supported Android versions, screen sizes, and known risk areas.

8.Can I use device farms in addition to human testers?

Yes. Android Developers points to Firebase Test Lab and similar services for testing on real devices and emulators that you may not physically have access to.

9.Does device diversity guarantee Google Production Access?

No. Device diversity can strengthen your testing process, but Production Access is separately reviewed by Google. Developers are asked about tester engagement, feature usage, feedback, and changes made based on testing.

10.How many devices should my 12 testers cover?

There is no Google-mandated device ratio. Use the testers you have strategically and prioritize environments relevant to your app's expected users.


About the Author

Saksham Goyal

Saksham Goyal

Expert software tester and contributor to the community. Passionate about AI, automation, and user experience.