In 2019 GEO Jobe updated it’s hosted services environment in AWS to ArcGIS Server 10.6.1. As part of the migration, we chose to migrate the files for our cached map services to Amazon S3 (Simple Storage Service). The ability to serve out cache files from Amazon S3 was first introduced in ArcGIS Enterprise 10.5. During our migration planning, we saw several potential benefits to moving the cache files to Amazon S3, namely:
- Potential cost savings by moving the cache files off of EBS volumes.
- Ability to move away from file shares (\\<host\folder\etc) configured for publishing.
- Simpler service management and publishing workflows.
Resources and documentation on how to set up Amazon S3 for map caches in your environment can be found here.
In Amazon S3 we have a bucket called ‘gpccache’ (internally, ‘GPC’ is the acronym we use for ‘GEOPowered Cloud’). This bucket contains one folder called ‘arcgiscache’ as specified in the documentation, and the arcgiscache folder contains all of the map caches. From a workflow perspective, we use the caching tools in ArcGIS Pro to create a cache locally and then upload the cache to Amazon S3 using the web interface or Cloudberry Explorer.
In each ArcGIS Server, a data store was created that references the cache location in Amazon S3. These are configured in Site > GIS Server > Data Stores > Register (Cloud Store). For our purposes we created a dedicated user in our AWS Management Console that is used for the authentication between our ArcGIS Servers and the Amazon S3 bucket storing the cache files. As such, we used the Access Key / Secret Access Key authentication method in our Cloud Store configuration seen below.
Once everything is in place, you should be able to choose the Cloud Store as the cache directory when publishing a cached service. Since we ‘pre-cook’ the cache files in ArcGIS Pro, we set the option to ‘Build cache manually after the service is published’ option in the publishing dialog so the files aren’t recreated.
As we’ve updated our cache location, we’ve picked up a few tips and tricks along the way:
- If you use folders to organize services in ArcGIS Server, the folder name needs to be part of the folder name in Amazon S3. For example, if you’re publishing a service called ‘cache2020’ into an ArcGIS Server folder called ‘NewYork’ the resulting folder name in your arcgiscache folder will be NewYork_cache2020.
- If publishing using an image extent feature class, make sure the projection of the feature class is in WGS84 Web Mercator (Auxiliary Sphere) 102100 (3857), and make sure the extent of the polygon covers the full extent of the source imagery.
- Occasionally, staging the service to a service definition and uploading the .sd file to ArcGIS Server manager is more successful than publishing with ArcMap.
- In the service properties, make sure the option to ‘Allow per request modification of layer order and symbology’ is disabled.
Overall, the transition to Amazon S3 for our cached file storage has been successful and we are glad that we made the move as it simplifies the management of this component of our cloud while providing excellent performance at a lower cost.
