Update – Sept 2021.
– Added a new switch “-mode” [Online | Offline]. Specifies whether you want to pull data live from the server or use existing CSV files.
– Fixed bugs and added more error handling
Intro
Keeping a clean and healthy environment is important when managing a large number of devices in Workspace ONE UEM. Over time duplicate and stale device records, especially for Windows 10, can grow and grow. Without regular maintenance, you could potentially have thousands of records that are no longer valid and will affect the reporting of your deployments. This happens due to normal attrition of devices, re-imaging, and device resets. I took an internal script developed by our VMware IT system admins and modified it for external use so that you can easily clean up device records using WS1 REST API. The script is available on Github: Start-UEMMaintenance.ps1. To download it, click “Raw” then right-click and “Save As”.
What the script do?
This script connects to your VMware Workspace ONE UEM environment and gets duplicates, stale records, or problematic devices (devices with invalid serials). It also can get duplicate user accounts. Once you run any of the “get” actions, it will save the data to several csv files located under C:\UEM-Maintenance\[uem server]\[today’s date]. If you run it again on the same day, it will search for valid CSV files first before reaching out again to the server. This is to improve speed, allow for editing of CSVs, and reduce load on the server. If it does not find a valid csv, it will go ahead and reach out to the server. Additionally, it asks for and stores the UEM credentials in an encrypted file (saved C:\UEM-Maintenance\ [uem server]\Logs\Creds.txt) with AES encryption. The key is saved C:\UEM-Maintenance\ [uem server]\Logs\AES.key. This allows the script to be run in an automated fashion by a service account or multiple users on the same internal server. However since the encryption key is on the same system, care must be taken on who can access the system this script is running from. This key can also be saved on a different location for improved security.
Security
Using REST API with Workspace One involves two things: first, you need an API key. This essentially “gets you in the door” and is tied to the Organization Group. However, to actually do anything, such as reading or deleting devices you still need a Workspace One admin account. This can either be a basic account or a service account synced in from Active Directory. The minimum this account needs is REST API access as well as “delete’ device records. The simplest way to achieve it is the grant the account “Airwatch Administrator” access. However, this gives it way more access than it needs so the best practice always is to create a least privileged access role. If you do set it to “Airwatch Administrator”, give it that role during the cleanup and then change the account to “read only” role. This is to ensure no one external ever get the API key and admin credentials.
Additionally, as mentioned above when you run the script the first time (per environment) it will prompt you for those admin credentials and then save them encrypted to C:\UEM-Maintenance\ [uem server]\Logs\Creds.txt. The key to unlock and retrieve the creds is in the same directory so this isn’t a true foolproof encrypted solution. It adds a layer of security but you still need to protect the AES.key file. You can do this several ways: First, delete the file after each time this script is run. If you do this then you will have to input credentials each time you run the script. Second, you can run this on an internal server where only legitimate admins have access. Or you can modify the script to store and retrieve this file from a different file share on your network.
With that out of the way, let’s dig into each function and how they work.
Functions
Get-DuplicateDevices
This function first gets all devices and saves them to AllDevices.csv. You can optionally search based on platform. Next, it will count the duplicates based on Serial Number and save to AllDuplicateDevices.csv. It loops through that list and sorts based on “Device Last Seen” and then by “Device ID” so that the most recent device (meaning the “good” device) is excluded. All of these exclusions are sent to DoNotDeleteList.csv. The rest are the true duplicates and should be deleted. This list is DuplicateDevicesToBeDeleted.csv and will be used by the “Delete-Duplicates” function. You can look at these files as well as edit them for accuracy as well as to reduce the final deletion list. I.e. as a test you could delete all rows except one in the DuplicateDevicesToBeDeleted.csv file so that only one device will be deleted. This is a good test to ensure your credentials work properly. Additionally, this can sometimes take some time depending on how large of an environment it is. This function supports two optional parameters:
-FilterSerial
It excludes devices with invalid serials as those may not be true duplicates. See below for more details on this.
-Platform
Specifies which platform type you’d like to search duplicates for. Valid options are: ‘Mac’, ‘Win10’, ‘Android’, ‘iOS’, ‘ChromeOS’, ‘Any’. Any is the default if no parameter is specified.
Delete-DuplicateDevices
This will delete all devices found in DuplicateDevicesToBeDeleted.csv. If that file is not there, it will then run Get-Duplicates.
Get-StaleDevices
This first looks for an AllDevices.csv file and will import it if found. Otherwise, it will run the “Get-DuplicateDevices” command and pull live from the server. Then after all devices are gathered, it will then sort and look for all devices not seen in the last 90 days (default). If you specify a different value in the “day” parameter, then you can adjust how far back you want to filter. After this it will save all stale devices to StaleDevices.csv. Optional parameter supported:
-Days
Specifies how many days back the script should check for stale records. Default is 90 days.
Delete-StaleDevices
This will delete all stale devices found in StaleDevices.csv. If that file is not there, it will then run Get-StaleDevices.
Get-ProblematicDevices
This first looks for an AllDevices.csv file and will import it if found. Otherwise, it will run the “Get-DuplicatesDevices” command and pull live from the server. Then after all devices are gathered, it will then sort based on the $SerialFilter variable at the top of the script. At the time of this writing, I have it set to filter any of these values
'System Serial Number',
'To be filled by O.E.M.',
'Default string',
'',
'0',
'1234567'
Devices can have invalid serials for a number of reasons and need to be looked at why the inventory is not working properly. This can sometimes indicate a bad or incomplete enrollment. This function will save all problematic devices found to ProblematicDevices.csv
Delete-ProblematicDevices
This will delete all problematic devices found in ProblematicDevices.csv. If that file is not found, it will run Get-ProblematicDevices.
Get-DuplicateUsers
This searches the environment for duplicate users. Duplicate users can exist for a variety of reasons, but the most common is creating basic accounts with the same username at different OGs. You can filter based on user type (Basic, Directory, or Any) or a user list (csv file). Optional parameters supported:
-Userlist
Specify path to a csv file that has a list of usernames (command separated)
-Usertype
Specifies type of search to search for (basic, directory, or Any)
Example:
.\Start-UEMMaintenance.ps1 -server myserver.awmdm.com -ApiKey “n0t@re@lK3y” -Action Get-DuplicateUsers -UserList C:\temp\userlist.csv -UserType ‘BasicOnly’ -UserList C:\temp\userlist.csv
Required Parameters
-Server
Specifies the server you are targeting in format myserver.awmdm.com
-Apikey
Mandatory parameter for the API key that is required for the script to connect via REST API to your server. These keys are per OG and are found under All Settings > System > Advanced > API > REST API.
-Action
Mandatory parameter that specifies the action the script should take. Options are: ‘Get-Duplicates’, ‘Delete-Duplicates’, ‘Get-Stale’, ‘Delete-Stale’, ‘Get-Problematic’, ‘Delete-Problematic’.
Examples
For more examples, see the readme documentation here.
Hi Brooks,
an alternative solution to improve security, instead of storing credentials in an encrypted file, is using API/system/admins/session. It provide a temp tenant key and an access token. That way, username and password will never be stocked anywhere.
It is available starting from Workspace ONE UEM 1907 release (waiting for Oauth authN which should be released with 2001).
best regards,
Hi,
I have some WIN10 devices that are not seen for 1 day in my WS1 Console (prepared specially for testing this script). When I run the script to get the stale devices with parameter “-Days 1” the file Win_StaleDevices.csv is created and populated with the correct devices (devices that do not communicate with UEM for 1 day) . After that I´ve tried to leave only one device in the Win_StaleDevices.csv file and execute the script with option “-Action Delete-Stale” to check if I the user has proper delete permissions but the scrit fails. This is the output:
“Start Log
Getting encrypted credentials…
Checking if export folder C:\UEM-Maintenance\xxxxxxx.awmdm.com\Win_2020-05-06 exists.
Folder C:\UEM-Maintenance\cn556.awmdm.com\Win_2020-05-06 exists
Looking for existing csv of stale devices…
C:\UEM-Maintenance\cn556.awmdm.com\Win_2020-05-06\Win_StaleDevices.csv found!
Importing Win_StaleDevices.csv
1 devices found.
0 devices have not communicated with UEM since “2020-05-05 (90 days)”.
0 stale devices exported to “C:\UEM-Maintenance\cn556.awmdm.com\Win_2020-05-06\Win_StaleDevices.csv”.
Inputdata parameter is null in C:\Temp\Script\Start-UEMMaintenance_Windows.ps1: 546 Carácter: 51
+ … tions = Format-DevicesForDelete -InputData ($global:StaleToBeDeleted)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Format-DevicesForDelete], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Format-DevicesForDelete
Delete-DevicesFromUEM : No se puede enlazar el argumento al parámetro ‘InputData’ porque es nulo.
En C:\Temp\OtroScript\Start-UEMMaintenance_Windows.ps1: 547 Carácter: 36
+ Delete-DevicesFromUEM -InputData $deletions
+ ~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Delete-DevicesFromUEM], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Delete-DevicesFromUEM”
I´ve also tried to set the $days parameter in the script to “1” but it is still not working.
Does your script should work the way I´m trying or it is not possible to delete devices not seen for 1 day? Am I doing something wrong?
Thanks for your script and for your time.
Regards
Hi,
I´ve been further ivestigating the problem I mentioned yesterday about the Delete Stale function. I´ve found out that everytime I open the Win_StaleDevices.csv file and delete some records corresponding to devices I don´t really want to delete in spite of corresponding to Stale machines the execution crushes with the message I showed you yesterday.
If i leave the Win_StaleDevices.csv untouched everythig works perfectly. I´ve tried with 1, 3 or 8 days as stale period and everytime it has work. I don´t know if the Win_StaleDevices.csv file can really be edited.
A great feature for your script would be the possibility of reducing the scope of searchs/actions to specific organization groups.
Again, ank you for your script and your time
Hi Rafael,
I can take another look at the script to see what’s going on. Are you deleting entire rows?
Brooks
I have tried running this on multiple Workspace One instances using the Get-Duplicates and Get-Stale commands but both yield zero devices when ran. I verified the API key and it appears to be correct and working as if I put in an invalid one I get an error as expected. Looking at the logs it creates all the files and behaves as it should other than yielding zero results. Any ideas?
Hi Jeremy – What version of the console are you on? There is a bug in 2003 that can prevent data from getting properly returned due to a data error on the database side. It is supposedly getting fixed in 2006. If you look at line 475 of the script, you will see that the pagesize is set to 30,000 and will loop until there are no more devices. You can change that to 500, 1000, etc to see if you get data back.
Thanks for the reply, I am running version 2003, I tried your suggestion unfortunately setting the list down to even as low as 10 devices didn’t yield and results for Get-Stale or Get-Duplicates. Do you have a link or anything for this bug? This may impact some other systems we use and I want to get more details on it. Thanks for you help.
First, thanks for the great work Brooks. Just tried this script on a 10000+ device environment and successfully deleted 87 duplicate devices. Added additional criteria for $SerialFilter and that worked well too. Just a bit of feedback on trying out “Delete-Duplicates” function.
Quoiting this from the write up. “You can look at these files as well as edit them for accuracy as well as to reduce the final deletion list. I.e. as a test you could delete all rows except one in the Win_DuplicateDevicesToBeDeleted.csv file so that only one device will be deleted.”
The script seems to always referring to “Win_AllDuplicateDevices.csv” and regenerating “Win_DoNotDeleteList.csv” and “Win_DuplicateDevicesToBeDeleted.csv” on every run. So, any changes to “Win_DuplicateDevicesToBeDeleted.csv” for testing purposes or to exclude some devices doesn’t work. So, when I tried to just keep one device in the file and tried to run Delete-Duplicates, all the duplicates are deleted. No harm done as I have already checked the list and built custom $SerialFilter.
Perhaps, the modifications should be done on “Win_AllDuplicateDevices.csv” instead for limited testing? I haven’t tried it. But that should work based on the script output I am seeing.
Like Brooks above, returns 0 results. This looks like it would be ideal for us, but so far not working 🙁 Any additional idea why that may be?
EDIT: Didn’t realize this was limited to Windows devices, should have clued in based on name. Removing platform=WinRT& from Line 475 gave me what I needed! Thanks
Thanks Dmitri, this was my issue too !
Dmitri – I have made several enhancements to the script, one of which is the ability to filter platform type so it’s not just a win10 thing.
Hi thanks for sharing this, very useful!
I’m looking for a simple script to simply delete devices in bulk via API. Ideally I would like to filter by device tag & enrollment status to only show devices that are unenrolled. Is there a way to accomplish this quickly?
You can filter based on unenrolled status in the console by going to device list view, clicking filter and then selecting Status > Enrollment Status. Unfortunately, you can’t filter based on tags unless you create a smart group with that tag as membership filter. There is an API that can get a device list based on tag ID – https://cn1380.awmdm.com/api/help/#!/apis/10002?!/Tags/Tags_DevicesForTag
Hi! Thanks for creating these scripts and making them available to us in the community! I’m having an issue with the “Delete-StaleDevices” action against and existing “StaleDevices.csv” located in the proper location. The error is as follows:
Start UEM API REST method – POST (Deleting devices).
An error occurred: Error Type: System.Net.WebException, Error Code:403, Error Description:
Finish UEM API REST method – POST (Deleting devices).
Any ideas how to resolve?
Thanks!
I’ll have a to take a look at the code to ensure the API hasn’t changed. Have you been able to successfully delete with other functions? If not, you may want to double check permissions on the API account you are using.
Hi Brooks, thanks for your reply! I have not had success with other delete device functions, but the other attempts were with someone else’s script (I only need to delete devices old devices, not duplicates or users). I’ll check permissions, but the credentials I’m using are my own and I’m a console admin so didn’t think that would be the issue (but you never know). BTW I’m using console version 2102 and our environment is dedicated SaaS. Thanks again!
Hi Brooks,
I was able to get this up and running for our UEM environment. However, I have a question regarding Get-DuplicateDevices.
When I run it once, it does exactly how it should.
But, when I run it again (on the same day) the data from “AllDuplicateDevices.csv” is removed and replaced with only : ÿþ
In turn, this results in DoNotDeleteList.csv and DuplicateDevicesToBeDeleted.csv going blank.
If I delete all the CSV’s except AllDevices, it works as expected again.
Any ideas?
Jordan – Yes, this is as designed. I did it as I figured throughout the same day there isn’t going to be much change and so the initial load of the data would be fine. It just saves time so you don’ have to download every device from the console each time you run it. I may add an “online” or “offline” switch to give you control over that.
Brooks, now that WS1 has a bug which forces you to delete a device before you reimage it, can the script be updated so you can get duplicate devices for a supplied device name or serial number, so that you can delete the one device before reimaging it and re-enrolling it?
You have to completely delete the device record before re-enrolling the device? That sucks. I supposed the script could be modified to look for duplicates for just a single device but I’m not sure how that will help you in this case since you still have to delete all the records?
Can the script be updated to search for a single device by name or serial, to use to remove a device reimage/re-enrollment?
So working with your script today, I wanted to remove stale devices. I pulled the current list of stale devices, and then went back and removed 122 rows which we deem as hold devices, cannot be removed at this time, and resave the list in the original file.
.\Start-UEMMaintenance.ps1 -server servername -ApiKey “apikey” -Action Delete-StaleDevices -Days 60 -Mode Offline
I used this command line with our server name and apikey , using offline mode so it would use the local .csv file. For initial testing I limited the stale device file to 12 devices. When the script ran it showed that it found 12 devices in the file, but then said 0 files had not communicated in 60 days and exported 0 devices to the staledevices.csv, overwriting what was there with a blank file.
this is the error log
Start Log
Getting encrypted credentials…
Checking if export folder C:\UEM-Maintenance\as274.awmdm.com\2022-03-04 exists.
Folder C:\UEM-Maintenance\as274.awmdm.com\2022-03-04 exists
Looking for existing csv of stale devices…
C:\UEM-Maintenance\as274.awmdm.com\2022-03-04\StaleDevices.csv found!
Importing StaleDevices.csv
12 devices found.
0 devices have not communicated with UEM since “2022-01-03 (60 days)”.
0 stale devices exported to “C:\UEM-Maintenance\as274.awmdm.com\2022-03-04\StaleDevices.csv”.
Format-DevicesForDelete : Cannot bind argument to parameter ‘InputData’ because it is null.
At C:\UEM Maintenence Script\Start-UEMMaintenance.ps1:795 char:51
+ … tions = Format-DevicesForDelete -InputData ($global:StaleToBeDeleted)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Format-DevicesForDelete], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Format-DevicesForDelete
Devices to be deleted: 0
Are you sure? Please enter ‘y’ or ‘n’.: n
Can you tell me what the problem is. I added the -Days verb because on the first test it said it was looking for devices 90 days old, so I put in -Days 60 because this is the parameter I used to the stale device list originally.
To help whoever comes along to see this, I finally found the issue. IN the Get-StaleDevices function, the portion that is filtering devices is the problem, and it took me a while to understand why, even though it was staring me in the face the whole time. The $date variable gets pulled in and formatted as yyyy-MM-dd at the top of the script. Problem is in this function, and probably in other portions of the script, the date coming in from the csv file is formatted dd-MM-yyyy. This is why my CSV file kept erasing itself. SMH. For my fix, I created a temporary variable in the foreach loop and converted it to yyyy-MM-dd format, to make the least amount of code changes. Once I did that, the script filtered the data as expected. Now to go test it against my environment!
Good find. I haven’t had a chance to take a deeper look at this but if other have this problem in the meantime, they can use your workaround. I’m curious if something changed on the server-side since I originally wrote it…
Now that this piece is working, I get a 403 error when it actually tries to run the removal, and I’m not sure why. I have been give the highest access in the console, so this should work.
Exact issue, 403 when attempting to remove stale devices. Gonna see if I cant find anything in the script that stands out.
Hi,
Thanks for this powerfull script, unfortunately i also have the ERROR 403 when i want to bulk delete.
did you have a chance to take a look on this?
i checked the API rest URL’s on VMWare but it seems thez are not changed. strange.
thanks a lot