Downloaded 98 times
Uploaded byJohn Reiser
Python and GIS: Improving Your Workflow
The document discusses the advantages of using Python with GIS, particularly ArcGIS, for automation and enhanced functionality. It highlights the integration of Python within ArcGIS, including field calculations, custom tools, and the ability to extend ArcGIS capabilities. Additionally, it covers topics like ETL processes, working with databases, and resources for learning Python in the context of GIS.
More Related Content
![[APIdays NY] Managing the usage of Asynchronous APIs: What does it take?](https://cdn.slidesharecdn.com/ss_thumbnails/apidays-2021-async-210809082511-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to Python and GIS: Improving Your Workflow
Python and GIS: Improving Your Workflow
- 1. Python and GIS ImprovingYourWorkflow DVRPC- IREG 9 December 2015
- 2.
- 3. Automation "HollerithMachine.CHM" by AdamSchuster - Flickr: Proto IBM. CC BY
- 4.
- 5. ArcGIS & ModelBuilder •Limited iterative capacity • Limited reusability • Limited to ArcGIS • No version control
- 9. Why program inPython? • Extensible nature of the language gives you the flexibility to change it as you see fit. • Conversely, others may have already worked on a similar problem and released their solutions on the web for you to incorporate. • Support for various OSs and GISs means that your tools can be easily migrated to other platforms and maintained as GIS evolves. • A strong community supports the software and its continued development.
- 10. From the ArcGISperspective • You might have already used Python within ArcGIS. • Python can work withinArcGIS – – Custom field calculator functions – Scripts within ArcToolbox and PythonToolboxes – Interactively through the Python prompt • Python can extendArcGIS functionality – – Enable scheduled tasks – Improve upon ETL tasks – Talk to other software or components
- 11.
- 12. Python within ArcGIS •Python is integrated with the ArcGIS Desktop software and can be accessed in several ways: – Field calculator – ArcGISToolbox scripts & PythonToolboxes – Python Add-ins – Interactive Prompt • Understanding Python can help with many operations. • Your level of knowledge can vary; expanding your knowledge will only open up more opportunities.
- 13. The Field Calculator •The Field Calculator (and Calculate Field tool) allow you to programmatically change the values within a layer's or table's column. • Python can be used to calculate new data from existing fields or other properties.
- 14. Referencing Other Fields •Fields are delimited with exclamation marks in the Python code block. • !block! refers to the Block column. • A function can be defined to do additional processing, with fields passed as parameters. • Creating a PAMS_PIN for NJ parcel data.
- 15.
- 16. The SHAPE Field •The SHAPE field has several methods that can be accessed within FieldCalculator to retrieve attributes of the record's geometry. • !shape.area!, !shape.length! • Unit type is modifiable: !shape.area@SQUAREYARDS! • !shape.partCount! • !shape.pointCount!
- 17.
- 18. Great Starting Point •Exporting from ModelBuilder is a great starting point for many projects. • Declare variables/layers/workspaces using ModelBuilder UI. Start to put the pieces together. • Export an incomplete model to Python for final development. • The shell will be there – it will produce a valid (though potentially nonfunctional) arcpy script. • You can then complete and extend the functionality.
- 21. The arcpy Module •Importing the arcpy module provides access to the ArcGIS Engine in Python. • Functionality provided by arcpy includes the tools provided by ArcToolbox, as well as some specialized functionality such as cursors. • Easy, pythonic way to work with ArcGIS. import arcpy arcpy.AddField_management( "c:/data/Portland.gdb/streets", "LENGTH_MILES", "TEXT") arcpy.CalculateField_management( "c:/data/Portland.gdb/streets", "LENGTH_MILES","!shape.length@miles!", "PYTHON_9.3")
- 22. Standalone arcpy Scripts •arcpy can be called from any Python program that is using the version of Python installed with ArcGIS. • Commonly underWindows, the version installed with ArcGIS is the only Python installed. • Python and arcpy provides access to geoprocessing functionality and tasks, with lower overhead. • Programs using arcpy can be run without having any of theArcGIS Desktop suite running. • Great for scheduled or background tasks.
- 23. arcpy Cursors • Cursorsenable you to iterate through a series of rows, insert new rows, or update existing rows. • Cursors empower you to perform analysis on the feature level, as opposed to the layer level. • Features can be directly accessed, enabling greater functionality than possible through most tools. • Features can also be modified individually.
- 24.
- 25.
- 26.
- 27. Python Add-Ins • At10.1, ESRI introduced Python Add-Ins. • ReplacingVBA as the method for creating UI tools. • ESRI provides an Add-In Wizard to help create these tools. • ESRI training: "Creating Desktop Add-ins Using Python"
- 28. Download Imagery • NJOIT-OGIS'sWMS services are great, but what if you want the source tiles? • Python Add-Ins allow you to pull the data down, extract it and load it into ArcGIS. • Allows for interactive selection within a your map frame, with the data automatically added to the map once the download is complete.
- 30.
- 31.
- 34. PythonToolbox • PythonToolboxes arepure-Python files that appear as ArcToolbox toolboxes. • Tools are defined as subclasses. • A toolbox containing several independent tools can exist as one text-based file. • Great format for transparency and version control. • Uses .pyt as a file extension.
- 38.
- 39. Make ETL Easy •Extract, transform, load processes can be tedious, especially if they are complex and frequent. • Python can help assist you with bringing in outside data, conforming it to your needs and loading it into your database/storage. • Python can connect to the web and databases, as well as process and convert GIS data. • Python can also be run from Scheduled Tasks/cron, enabling you to fully automate these operations.
- 40. Fiona • Fiona isa Python wrapper for the GDAL/OGR library. • https://pypi.python.org/pypi/Fiona • Enables you to convert/project/process a multitude of spatial data formats. • Expanding on the previous example, the downloaded data can then be processed, converted to/from a specific format, reprojected, etc.
- 41. Web Retrieval &Extraction • Several modules exist to enable easy downloading of remote resources. • urllib (plus urllib2, urllib3, httplib, etc…) • zipfile is a module for working with PKZIP files. • Let's look at some code to automate downloads and extraction.
- 42.
- 43. psycopg2 • psycopg2 moduleprovides a Pythonic way of querying a PostgreSQL database. • PostgreSQL-specific, enables accessingArcSDE or PostGIS-formatted data. • Full, secure access to your databases. • Very easy to use the records returned from a database query in your Python programs.
- 47.
- 53.
- 54. Books • Lutz, Mark.Learning Python, 5th Edition. O’Reilly Media. ISBN: 978-1-4493-5573-9 • Lubanovic, Bill. Introducing Python, 1st Edition. O’Reilly Media. ISBN: 978-1-4493-5936-2 • Shaw, Zed. Learn Python the HardWay. Addison- Wesley. ISBN: 978- 0321884916
- 55. OnlineTutorials • Learning Python: –Code.org – Codecademy • ESRITraining offerings: – Python for Everyone - http://training.esri.com/gateway/index.cfm?fa=catal og.webcoursedetail&courseid=2520 – Using Python in ArcGIS Desktop 10 - http://training.esri.com/gateway/index.cfm?fa=catal og.webCourseDetail&courseID=1868
- 58. Online Communities • Blogs –geospatialpython.com – sgillies.net – many more… • Stack Exchange – gis.stackexchange.com – stackoverflow.com • GeoNet - Python & Geoprocessing forums – http://forums.arcgis.co m/forums/117-Python
- 59. Thanks for listening! •Feel free to follow up with questions: jreiser@njgeo.org @johnjreiser onTwitter github.com/johnjreiser
Editor's Notes
- #26 https://arcpy.wordpress.com/2012/11/15/shifting-features/