I distinctly recall the first thing I learned upon entering my career after school. The harsh reality I encountered was that things were never as tidy they were in my GIS lab courses; data wasn’t always perfectly manicured and I didn’t always have the functionality I needed at my fingertips. Big surprise, right?
A recent project was no different, due to project constraints and business requirements, I had to create a solution from a collection of applications and tools in order to meet the needs of our staff. The project was to provide a solution for the Fire & Rescue Services Staff where they could record survey responses during their annual Home Smoke Alarm Program. This type of program is something common among Fire Departments and involves meeting residents, carrying out a survey, and conduct fire prevention education.
The specifics of our scenario was that the solution needed the following:
It had to be map-centric. The existing process revolved around a map showing pre-selected properties that needed to be visited, and really why wouldn’t you want a map? -> I had never used Explorer before but that’s obviously a map.
The survey had to be easy to use. It also needed to have data validation and be concise. -> Perfect, I’ve used Survey123 a bunch and I know it can do that!
The progress of the program needed to be monitored. Coordinators, Chiefs, and administrative staff needed to have an interface to review results from the program, monitor progress, and trigger emails for follow up actions. -> No problem, we can put the results in a Dashboard and it will have ALL the gauges.
The solution needed to be disconnected in the field. -> Say what?
Now there has been a lot of progress in the capabilities of taking field apps offline but it can still be a scary requirement, stuff usually just gels better together when it’s connected. Regardless, I carried on. From here on I’ll be breaking the solution into the individual components and outline the high-level process.
Explorer
I quickly found that the functionality differs significantly between iOS and Android versions of the Explorer application and I was destined to be bound to the Android version. In addition, at the time I built this solution in 2018, downloading maps for Offline use in Explorer wasn’t very straight-forward (in the way it is in Collector).
Map Packages in ArcGIS Pro
I hadn’t thought I’d need to use Pro but in order to have Explorer download map packages for offline I had to create them locally. THis task consisted of creating a basemap layer and then adding my selected inspection addresses from a layer I'd published to ArcGIS Online. Using ArcGIS Pro to author the packages enabled several advantages:
I was able author my own custom basemap for the map packages.
I could link features from my Explorer Map Package to my Survey123 using a custom URL scheme in the feature Pop-ups. I also used this scheme to pre-populate the address UID.
And I could automate the publishing process using Python. I could get a little crafty with the scripted process…
Scripty Business
I published the pre-selected addresses for the program to ArcGIS Online. Along with a uniqueID for each address and the schema to contain responses from the survey questions, there was a SURVEY_STATUS field to indicate whether the address had been visited and a STATION_PLATOON field to indicate which platoon was responsible for visiting the address.
I added this layer from AGOL into my ArcGIS Pro map package twice and restricted one layer to only show COMPLETED surveys and the other INCOMPLETE (*only the INCOMPLETE addresses had a Pop-up link to the Survey123, this prevented duplicate entries). I also constrained both of these layers to a specific STATION_PLATOON (this removed clutter on the maps).
As my Python script iterated through a list of the possible STATION_PLATOON values it would:
Update the definition query of the layers to only show those selected addresses that belonged to the corresponding STATION_PLATOON.
Publish a Map Package with the name “Smoke Alarm Inspection Map: {STATION_PLATOON}”. This was shared to an AGOL organization group that had access to these maps.
The end result was a script that refreshed a collection of individually named map packages containing data layers from AGOL, had connections to a Survey123 app, and could be downloaded and taken offline by Explorer. *Full code sample is at the end...
Survey123
The Survey component was definitely the most straightforward. I created a survey using the Survey123 Connect and built in some data validation and pre-populated an ADDRESS_ID attribute using the UID from the layer of pre-selected addresses with the Pop-up embedded in the Map Package. I published this survey to my AGOL organization and was ready to go.
One thing worth mentioning, is that Survey123 creates it’s own supporting feature layer with a spatial point geometry associated with the location of survey regardless of whether there is a location collection question in the survey. Knowing our devices were going to be offline I didn’t want to present these points as ‘fact’ when pushing the results into a Dashboard. Therefore, I had to use some background processing to reconcile the collected data with our pre-selected addresses.
Enter FME
The final piece to the puzzle was to reconcile the survey results with our addresses. This would ensure results would be associated with valid addresses in our data (meaning staff didn’t need to enter address information prior to performing a survey). In addition, all of the collected data would be represented in the correct location on the map despite inaccurate disconnected location services on the device.
To achieve the reconciliation we used Safe Software’s FME to essentially perform a join between the feature layer from Survey123 and our pre-selected addresses. Once joined, the responses collected during the survey were transferred into the schema of the address points, just like a join and Field Calculate). A few other tasks were also performed such as, triggering an email if follow up was required and some basic data validation to ensure an address wasn’t visited twice in one day.
*This step could likely also be achieved using another Python script if FME isn’t in your toolbox.
Now make it all work together
Now that everything existed separately, the final step was to tie everything together and create a dashboard to show the results. Both the Python script to publish map packages and the FME process to reconcile survey results executed nightly. This meant that after every shift when staff returned to a wifi connection and submitted their survey responses to AGOL the results were reconciled with the selected addresses and the status was updated. The status then changed how those locations were represented within the map packages that were republished by Pro and staff would download a refreshed map package before heading out the next day. This kept progress relatively up to date and was straight forward for the users.
So far, this solution has worked out excellently for our users will be entering it’s second year of use this year. Despite not having every imaginable feature or capability at our fingertips, there are enough bits and pieces out there to assemble some really clever solutions and deliver a lot of functionality with a little ingenuity and effort. So don’t get stumped when an application is missing something, it’s likely you can integrate something with a little poking and prodding while gaining new skills along the way. It may not always look pretty, but really is anything ever perfect?
Good luck!
Code Sample:
#------------------------------------------------------------------------------- # Name: SA_Map_Packer.py # Purpose: Automate the packaging of mobile map packages from ArcGIS Pro and # upload them to AGOL to provided updated maps to Explorer application. # # Author: wdavison #-------------------------------------------------------------------------------
import arcpy
arcpy.env.overwriteOutput = True
# Output location for map packages output_location = r'OUTPUT_DIRECTORY'
# Function to update the definition query for a layer def update_definition_query(layer, layer_status, station_platoon): # update definition query definition_query = """INSPECTION_STATUS = '{0}' AND STATION_PLATOON = '{1}'""".format(layer_status, station_platoon) # Set the definition query on the layer layer.definitionQuery = definition_query
# Function to refresh the map package and upload it to AGOL def refresh_map_pack(input_map, station_platoon, station_platoon_list): #,mmpk,title,summary,description): # Output location/name for mobile map package proper_name = 'Station_{0}_Platoon_{1}'.format(station_platoon.split("-")[0], station_platoon.split("-")[1].replace("_Aer", "_Aerial")) ## slight change here to new format denoting station platoon... mmpk = '{0}\{1}'.format(output_location, proper_name)#station_platoon.replace("-", "_")) # Title of the map pack, how it appears in explorer* title = 'SA Map {0}'.format(station_platoon) # Summary summary = 'Smoke Alarm Inspection Map {0}'.format(station_platoon) # Description description = 'Map package containing addresses assigned to Station/Platoon {0} for smoke alarm inspection. These' \ 'address points have a pop-up configured to activate a Survey123 inspection on a mobile device.'
# Create mmpk print('Refreshing map package {0} | {1} of {2}...'.format(station_platoon, station_platoon_list.index(station_platoon) + 1, len(station_platoon_list))) print('...Creating map package...') arcpy.management.CreateMobileMapPackage(input_map, mmpk, None, None, "-8975298.81309241 5377879.4995742 -8957576.91414757 5393268.99384716", "SELECT", title, summary, description,"Fire, Smoke Alarm, Inspections", None, None, "STANDARD") print('...Map package completed.')
# Upload package print('...Uploading map package.') arcpy.management.SharePackage(str('{0}.{1}'.format(mmpk, 'mmpk')), "USERNAME", "PASSWORD", summary, "Fire,Mobile Map Package,2D,ArcGIS Pro,mmpk", None, "MYGROUPS", "Fire Rescue", "MYORGANIZATION") print('...Package uploaded.') print('Refreshed map package {0}.'.format(station_platoon))
# Script Proper print('Running SA_Map_Packer.py') # Smoke Alarm Inspection Station-Platoon combinations station_platoons = ['1-A','1-B', '1-C', '1-D', '2-A','2-B', '2-C', '2-D','3-A','3-B', '3-C', '3-D','4-A','4-B', '4-C', '4-D', '1-A_Aer','1-B_Aer', '1-C_Aer', '1-D_Aer']
# ArcGIS Pro project aprx = arcpy.mp.ArcGISProject(r'{0}\SmokeAlarmMapPack.aprx'.format(output_location)) print('Output location: {0}'.format(output_location)) print('Map project: {0}'.format(aprx))
# Get the first map in the project mp = aprx.listMaps('SA_Map')[0]
# Get collection of layers in the project that contain "Smoke Alarm Inspections" layers = mp.listLayers('Smoke Alarm Inspections*') print('Layers: {0}, {1}'.format(layers[0], layers[1]))
# Iterate through Station Platoon combos for sp in station_platoons: # Update completed inspections update_definition_query(layers[0], 'Complete', sp) print('{0} definition query updated with {1}.'.format(layers[0], sp)) # Update pending inspections update_definition_query(layers[1], 'Incomplete', sp) print('{0} definition query updated with {1}.'.format(layers[1], sp)) # Refresh the map pack file refresh_map_pack(mp, sp, station_platoons) print('Maps refreshed.')
del aprx