Learn how to manage Google Play Closed Testing across different Android devices, versions, screen sizes, networks, and performance levels for better app testing.
One tester has a Samsung.
Another uses a Pixel.
Someone has a OnePlus.
Another has an older Xiaomi.
And then there is the person testing your app on a tablet that you completely forgot existed.
Welcome to Android app testing.
Device diversity is one of the biggest advantages of a real-user Closed Test, but it can also make testing harder to organize. Different Android versions, screen sizes, manufacturers, hardware capabilities, network conditions, and device performance can expose problems that never appear on the developer's own phone.
Google recommends using Closed Testing to gather targeted feedback from a wider set of testers before production. Android's developer guidance also warns that testing only on a single device can allow problems on other screen sizes and form factors to go unnoticed.
The solution is not to test every Android device on Earth. That would be a wonderfully terrible use of everyone's time.
Instead, build a balanced device-testing strategy.
Why Different Android Devices Matter
An app can behave perfectly on one phone and still have problems elsewhere.
For example:
A button may be cut off on a smaller display.
A layout may look strange on a large screen.
A feature may behave differently on another Android version.
A slower device may expose performance problems.
A manufacturer-specific implementation may affect notifications or background behavior.
A weak network connection may expose loading problems.
Different screen densities can affect UI rendering.
Android's guidance specifically notes that problems can go unnoticed when development relies on a single device and recommends testing across different screen and window sizes.
That is exactly why having testers with different devices can be valuable.
1. Don't Give Every Tester the Same Phone
If all 12 of your testers use the same phone model, you may get 12 opinions about one device environment.
That is useful, but limited.
A more diverse group might look like:
Tester
Brand
Device Type
Android
Main Focus
T01
Pixel
Recent
Core features
T02
Samsung
Galaxy
Recent
UI + navigation
T03
OnePlus
Phone
Recent
Performance
T04
Xiaomi
Redmi
Recent
Compatibility
T05
Motorola
Phone
Older
Performance
T06
Samsung
Galaxy
Older
Android compatibility
T07
Pixel
Recent
Notifications
T08
Xiaomi
Redmi
Recent
Network behavior
T09
OnePlus
Phone
Recent
Login
T10
Samsung
Large display
Recent
UI
T11
Android tablet
Tablet
Recent
Large screen
T12
Other
Phone
Recent
General testing
You do not need exactly this combination.
The principle is what matters:
Different testers should expose different environments whenever practical.
2. Track Device Information
Don't simply record:
Rahul is testing.
Record:
Rahul → Samsung Galaxy S23 → Android 15
Your tester tracking sheet can include:
Tester ID
Google Account
Device manufacturer
Device model
Android version
Screen size
Testing focus
Opt-in date
Feedback
Current status
For example:
Tester
Device
Android
Focus
Status
T01
Pixel 8
15
Core features
Active
T02
Galaxy S23
14
UI
Active
T03
OnePlus 12
15
Performance
Active
T04
Redmi Note
14
Compatibility
Active
This makes bug investigation much easier.
If three testers report the same issue and all use similar devices, you have an important clue.
3. Test Multiple Android Versions
Device diversity isn't only about manufacturers.
Android versions matter too.
A feature can behave differently across Android versions because of differences in APIs, permissions, system behavior, or platform restrictions.
Try to understand which Android versions your target audience actually uses.
Then make sure your tester group provides useful coverage across those versions.
For example:
Recent Android version
Test:
Latest platform behavior
New permission behavior
Modern UI
Recent system restrictions
Older supported Android version
Test:
Compatibility
Performance
Permissions
Layout
Core functionality
Do not deliberately test unsupported Android versions just to make your spreadsheet look impressive.
Test the versions your app is actually intended to support.
4. Test Different Screen Sizes
Screen size is one of the easiest ways for UI problems to hide.
A screen that looks perfect on your development phone might look completely different on:
Small phones
Large phones
Foldables
Tablets
Landscape layouts
Split-screen environments
Android's official guidance recommends testing different screen and window sizes because single-device development can allow layout problems to go unnoticed.
Ask testers to check:
Text wrapping
Buttons
Images
Cards
Navigation bars
Dialog boxes
Forms
Keyboard behavior
Scrolling
Orientation changes
A particularly useful instruction is:
“If anything looks cut off, overlapping, unusually small, or difficult to tap, take a screenshot and report it.”
Screenshots turn vague complaints into actionable bugs.
5. Test Both Fast and Slow Devices
Not everyone owns a flagship phone.
Your app may feel incredibly fast on a high-end device but frustratingly slow on a lower-end phone.
Ask testers to observe:
Startup time
Screen transitions
Loading indicators
Scrolling
Image loading
Search
Login
Background operations
Memory-related problems
Battery impact
You don't need to benchmark every phone scientifically.
You need to discover meaningful differences in real-world use.
6. Test Different Network Conditions
Your app doesn't live in a perfect Wi-Fi laboratory.
Testers may use:
Fast Wi-Fi
Slow Wi-Fi
Mobile data
Weak signal
Intermittent connections
Temporary loss of connectivity
Ask testers to try important workflows under different network conditions.
For example:
Login
What happens if the connection disappears during login?
Upload
What happens if the network becomes slow?
Payment
What happens if the connection drops during a transaction?
Content loading
Does the app show a useful loading state or simply freeze?
These scenarios can reveal serious usability problems.
7. Give Testers Device-Specific Tasks
Don't ask every tester to do everything.
Assign specific areas.
Group A: Different Android Versions
Focus on:
Permissions
Notifications
System behavior
Compatibility
Group B: Different Screen Sizes
Focus on:
Layout
Text
Navigation
Orientation
Forms
Group C: Lower-End Devices
Focus on:
Startup
Performance
Scrolling
Memory
Loading
Group D: Different Manufacturers
Focus on:
Notifications
Background behavior
Permissions
Device-specific issues
This creates broader coverage without asking every tester to repeat the same work.
8. Ask Testers to Report Their Device
When a tester reports:
“The app crashes.”
You need more information.
Ask them to provide:
Device: Samsung Galaxy A54
Android: Android 14
App version: 1.0.7
Feature: Checkout
Problem: App closes after tapping Pay
Steps:
Open cart
Select payment
Tap Pay
App closes
Now the development team has something useful.
A bug report without environment information is basically a mystery novel nobody asked for.
9. Keep the App Version in Your Bug Reports
Device information alone isn't enough.
Suppose:
Tester A reports a crash on version 1.0.6.
You release version 1.0.7.
Tester B reports the same issue.
Is it still happening?
You need to know which build they are using.
Record:
App version
Version code
Device
Android version
Feature
Steps to reproduce
Google Play distributes releases through testing tracks, and testers receive updates to the test version as new releases become available.
That makes version tracking important throughout the test.
10. Don't Assume a Device Problem Is an Android Problem
Suppose one tester reports:
“Notifications aren't working.”
Don't immediately conclude:
“Android notifications are broken.”
Investigate.
The problem could be:
App notification permission
Device settings
Battery optimization
App configuration
Network issue
Incorrect implementation
Manufacturer-specific behavior
Test the same feature on another device.
If several devices show the same issue, it is more likely to be an application-level problem.
If only one device or manufacturer shows it, investigate device-specific behavior.
11. Use Screenshots and Screen Recordings
Visual problems are much easier to diagnose when you can see them.
Ask testers to attach:
Screenshot
Screen recording
Error message
Relevant device information
For example:
Bug: Button overlaps text
Device: Redmi Note 13
Android: 14
App: 1.0.4
Screenshot: Attached
This gives developers immediate context.
12. Test the App in Portrait and Landscape
If your app supports orientation changes, test both.
Check:
Login
Forms
Video
Navigation
Dialogs
Lists
Images
Keyboard interaction
A screen that works perfectly in portrait can become unusable in landscape.
If your app intentionally supports only one orientation, make sure that behavior is deliberate and consistent.
13. Test Tablets and Foldables When Relevant
If your app is intended to work on larger screens, include larger-screen testing.
Android's testing guidance specifically covers different screen and window sizes and recommends testing across different form factors.
You don't necessarily need to buy every device.
For additional technical coverage, Android Developers points to services such as Firebase Test Lab for testing on specific real devices and emulators, including devices you may not physically own.
Real-user testing and automated/device-farm testing can complement each other.
14. Use Your Closed Test for Real-World Feedback
The purpose of Closed Testing is not simply to prove that the APK installs.
Google describes closed testing as a way to test pre-release versions with a selected group and gather targeted feedback before broader availability.
So ask testers to use the app naturally.
For example, if you're testing a shopping app, don't only ask them to open every screen.
Ask them to:
Search for a product.
Open product details.
Add something to a cart.
Modify the cart.
Complete the intended workflow.
Report anything confusing.
Realistic usage can reveal problems that scripted button-tapping misses.
15. Don't Confuse Device Diversity With Tester Quantity
Having 20 testers does not automatically mean you have good device coverage.
You might have:
20 testers
but:
18 identical Samsung models
That's still a narrow device sample.
Likewise, having 12 different devices does not automatically mean you have enough useful tester engagement.
Google's Production Access process asks developers to describe tester engagement, whether testers used available features, whether usage matched expected production behavior, and what feedback was received.
So think about both:
Tester quantity + Tester diversity + Tester engagement
16. Build a Device Coverage Matrix
A simple matrix can make your testing strategy much clearer.
Area
Device A
Device B
Device C
Device D
Login
✅
✅
✅
✅
Signup
✅
✅
—
✅
Payments
✅
—
✅
✅
Notifications
✅
✅
—
✅
Dark Mode
✅
✅
✅
—
Landscape
—
✅
—
✅
Poor Network
✅
—
✅
—
This lets you see where coverage is missing.
You don't need every tester to test every feature.
You need enough meaningful coverage to identify important problems.
17. Prioritize Important Features
Not every feature deserves the same amount of testing time.
For a typical app, prioritize:
High Priority
Signup
Login
Main user workflow
Payments
Data submission
Account management
Notifications
Critical API interactions
Medium Priority
Search
Filters
Profile
Settings
Secondary screens
Lower Priority
Rarely used features
Cosmetic settings
Non-critical animations
Your priorities depend on the app.
A payment app should not spend more testing time on a fancy animation than on transaction reliability. That would be a fascinating way to fail.
18. What to Do When Only One Device Has a Problem
Don't immediately ignore it.
Follow this process:
Step 1
Confirm the problem.
Step 2
Record the exact device and Android version.
Step 3
Ask another tester with the same or similar device to reproduce it.
Step 4
Try the same workflow on another manufacturer.
Step 5
Check whether the issue is reproducible.
Step 6
Investigate the root cause.
Step 7
Fix it if necessary.
Step 8
Release the updated build.
Step 9
Ask testers to retest.
This turns one unusual report into useful diagnostic information.
19. Retest After Fixes
Finding a bug is only half the job.
After fixing it:
Bug reported
↓
Fix implemented
↓
New build released
↓
Tester installs update
↓
Original device retests
↓
Different device retests
↓
Result documented
This is especially useful for device-specific issues.
A fix that works on a Pixel may still fail on a Samsung.
20. Keep Your Tester Spreadsheet Organized
A practical device-testing tracker might look like this:
ID
Tester
Device
Android
Network
Focus
App Version
Status
T01
Rahul
Pixel 8
15
Wi-Fi
Core
1.0.5
Active
T02
Priya
Galaxy S23
14
5G
UI
1.0.5
Active
T03
Amit
Redmi
14
4G
Performance
1.0.5
Active
T04
Neha
OnePlus
15
Wi-Fi
Login
1.0.5
Active
T05
Karan
Galaxy A-series
13
4G
Compatibility
1.0.4
Retest
This becomes extremely useful when reviewing bugs.
21. Use Play Console Alongside Your Own Tracking
Play Console is the official testing environment.
Your spreadsheet is your operational management tool.
Use Play Console for:
Testing tracks
Tester access
Releases
Distribution
Test feedback
Test configuration
Use your tracker for:
Device models
Android versions
Assigned tasks
Bug status
Test coverage
Follow-ups
Retesting
Google Play supports closed testers through email lists and Google Groups, and developers can provide a feedback URL or email address for testers.
22. Don't Try to Test Every Android Device
This is important.
There are far too many Android devices to test individually.
Instead, create representative coverage.
Consider:
Popular manufacturers among your audience
Supported Android versions
Low-end and high-end hardware
Small and large screens
Different network conditions
Relevant form factors
If your analytics show that 70% of your expected users have a particular type of device, prioritize that environment.
Testing should be based on risk and real users, not on collecting phone models like Pokémon.
23. When You Have Only 12 Testers
If you're completing the closed test required for Production Access, you may be working with a relatively small group.
In that situation, don't try to assign every possible device configuration.
Instead, distribute your testers strategically.
For example:
12 testers
3 Samsung
2 Pixel
2 Xiaomi
2 OnePlus
1 Motorola
1 other manufacturer
1 tablet or larger-screen device, if relevant
This is only an example.
Your actual distribution should reflect your app's target audience.
The objective is meaningful coverage, not a magic manufacturer ratio.
24. Prepare Testers Before They Start
Give testers clear instructions.
Tell them:
Your device: Please tell us your model and Android version.
Your task: Test login, navigation, and the main workflow.
What to report: Crashes, errors, slow loading, broken layouts, confusing screens.
What to include: Screenshot, device, Android version, app version, and steps to reproduce.
Google recommends clear tester instructions and a clear feedback channel for closed testing.
Good instructions produce better testing.
25. Use a Device Testing Checklist
Before finishing your test, verify:
Device Coverage
Multiple manufacturers tested
Supported Android versions tested
Different screen sizes tested
Relevant tablets/foldables tested
Low-end hardware tested where relevant
High-end hardware tested
Feature Coverage
Signup
Login
Main feature
Navigation
Notifications
Forms
Payments, if applicable
Error handling
Offline/poor-network behavior
Technical Coverage
Startup
Loading
Scrolling
Background behavior
Orientation
Memory/performance
Crashes
Feedback
Device information recorded
Android version recorded
App version recorded
Screenshots collected
Bugs documented
Fixes retested
Example 14-Day Device Testing Plan
Days
Focus
Day 1
Installation and device compatibility
Days 2–3
Signup, login and onboarding
Days 4–5
Core app features
Days 6–7
UI and different screen sizes
Days 8–9
Performance and network conditions
Days 10–11
Notifications and background behavior
Days 12–13
Bug fixes and regression testing
Day 14
Final device-specific checks and feedback
This is a practical framework, not a Google-mandated daily schedule.
The important thing is that your testing should reflect your app and its expected users.
Common Mistakes When Testing Different Android Devices
Mistake 1: Testing Only the Developer's Phone
The classic.
It works perfectly on your phone, therefore obviously the entire Android ecosystem must be fine.
Unfortunately, Android did not sign that agreement.
Mistake 2: Ignoring Older Supported Versions
If your app supports an older Android version, test it.
Mistake 3: Ignoring Screen Sizes
A beautiful UI on one phone can become a cramped disaster elsewhere.
Mistake 4: Not Recording Device Information
Without device details, diagnosing device-specific bugs becomes much harder.
Mistake 5: Treating Every Device as Identical
Manufacturer and hardware differences can matter.
Mistake 6: Testing Only Wi-Fi
Real users use mobile data and unreliable connections.
Mistake 7: Fixing a Bug Without Retesting the Original Device
Always verify the fix where the problem was discovered.
Mistake 8: Collecting Devices Instead of Feedback
Device diversity is useful only when it produces meaningful testing.
When Should You Use a Device Testing Service?
If you don't physically have access to enough devices, automated device testing can supplement your human testers.
Android Developers specifically points to Firebase Test Lab and similar device-farm services for accessing real devices and emulators that you may not own.
This can be particularly useful for:
Rare devices
Tablets
Foldables
Different screen sizes
Automated regression testing
Reproducing device-specific issues
Human testers and automated device testing solve different problems.
Human testers:
Real-world behavior and usability.
Device farms:
Broader technical coverage and repeatable tests.
Using both can give you stronger coverage.
How Different Devices Help With Production Access
For developers subject to Google's testing requirements, the Production Access application asks about tester engagement, feature usage, expected production behavior, feedback, and changes made based on testing.
That means your testing shouldn't simply be:
“We had 12 testers.”
A stronger process is:
“Our testers used the core features across different Android devices and versions. We identified several issues with navigation and performance, fixed them, and asked testers to retest the affected workflows.”
That demonstrates an actual testing process.
Be truthful and describe what actually happened.
Final Device Testing Strategy
A strong Android Closed Test does not require hundreds of phones.
It requires intentional coverage.
Think about:
Different manufacturers
Different Android versions
Different screen sizes
Different performance levels
Different network conditions
Realistic user behavior
Structured feedback
=
Better testing
Google Play's Closed Testing system is designed to help developers gather targeted feedback from selected testers before wider release.
Your goal is not to prove that your app works on one perfect phone.
Your goal is to discover where it doesn't work before your users discover it for you.
Final Checklist
Before completing your Closed Test:
Testers use more than one device model
Multiple manufacturers are represented where practical
Supported Android versions are covered
Different screen sizes are tested
Relevant tablets or foldables are considered
Lower-end devices are included when relevant
Wi-Fi and mobile data are tested
Core features are tested across multiple devices
Device-specific bugs are documented
Screenshots or recordings are collected
App version is recorded for each bug
Fixes are retested
Tester feedback is organized
Device coverage is documented before Production Access
Conclusion
Handling testers on different Android devices is not about forcing every tester to follow the same exact setup.
It is about using their differences to your advantage.
Track their devices and Android versions. Assign testing areas based on your app's risks. Test important workflows across different environments. Pay attention to screen sizes, performance, connectivity, and device-specific behavior. When a bug appears, record enough information to reproduce it and retest the fix.
Google recommends Closed Testing for targeted feedback before broader release, while Android's own testing guidance highlights the importance of testing across different screen sizes and form factors.
The best device-testing strategy is therefore not:
“How many phones can we test?”
It is:
“Which different environments could expose a problem before our users do?”
That is the question worth answering before you hit Production.
FAQs
1.Should all Google Play testers use different Android phones?
No. Google does not require every tester to use a different device. However, having useful device diversity can expose compatibility and usability issues that may not appear on a single device.
2.Which Android devices should I include in testing?
Prioritize devices and Android versions that represent your expected users. Include different manufacturers, supported Android versions, screen sizes, and performance levels where practical.
3.Should I test older Android versions?
Yes, if your app supports them. Test the Android versions that your application is intended to support.
4.Should testers use both Wi-Fi and mobile data?
For apps that depend on network connectivity, it is useful to test both. Poor or interrupted connections can expose loading, timeout, retry, and error-handling problems.
5.Should I test tablets?
If your app supports tablets or your target audience uses them, yes. Android recommends considering different screen and window sizes and form factors during testing.
6.What information should testers provide when reporting a bug?
Ask for the device model, Android version, app version, steps to reproduce, expected behavior, actual behavior, and a screenshot or recording when useful.
7.Do I need to test every Android device?
No. There are too many devices to test individually. Build representative coverage based on your target audience, supported Android versions, screen sizes, and known risk areas.
8.Can I use device farms in addition to human testers?
Yes. Android Developers points to Firebase Test Lab and similar services for testing on real devices and emulators that you may not physically have access to.
9.Does device diversity guarantee Google Production Access?
No. Device diversity can strengthen your testing process, but Production Access is separately reviewed by Google. Developers are asked about tester engagement, feature usage, feedback, and changes made based on testing.
10.How many devices should my 12 testers cover?
There is no Google-mandated device ratio. Use the testers you have strategically and prioritize environments relevant to your app's expected users.
About the Author

Saksham Goyal
Expert software tester and contributor to the community. Passionate about AI, automation, and user experience.