Reading and writing spatial data

Session 11

2026-04-15

Overview

Spatial data is everywhere
Navigating specifications, file formats, and services
Working with common spatial file formats
Working with popular spatial web services

Spatial data is everywhere

Shapefiles, GeoJSON files, and other spatial data files
Spatial data editing and hosting services, e.g. ArcGIS Online or Felt)
Open data portals serving spatial and non-spatial data, e.g. data.gov

Spatial data is everywhere

Google Maps, Apple Maps, Waze, and navigation apps (on your phone or in your vehicle)
Biking and running apps (Strava publishes a global user activity heatmap)
Web browsers on your phone and desktop application

Spatial data is everywhere

Phone camera photos (stored in the EXIF metadata accessible via exiftoolr)
Calendars (iCal), contacts (vCard), and emails (using IP address locations)

Creating spatial data is “easy”

GIS Desktop applications, e.g. QGIS
Web mapping applications (e.g. geojson.io, Felt)
Spreadsheet applications (e.g. Google Sheets)
Low-code data tools (e.g. Airtable or Microsoft Lists)

What can we do with it all?

Read and write spatial data files
Read and write related data files
Read from and write to web services

Standards and specifications

Open Geospatial Consortium (OGC) Standards are organized into categories:

APIs (e.g. Moving Features API)
Web Services (e.g. Web Feature Service)
Data Models and Encodings (e.g. GeoTIFF)
and more (Sensors, Publish-Subscribe, Syndication & Context, Discovery, Containers, Abstract Specifications)

Standards and specifications

Source: xkcd: Standards (CC BY-NC 2.5)

Common spatial file formats

Shapefile (.shp)
GeoJSON (.json, .geojson)
GeoPackage (.gpkg)
KML - Keyhole Markup Language (.kml)
Esri File Geodatabase (.gdb)
GPX - GPS Exchange Format (.gpx)

Common spatial file formats

These file formats vary in:

How geometry is stored
Input/output speed
File size
Compatibility with tools/systems
Other constraints or limits
And more!

Why are shapefiles bad?

Difficult to manage multiple files
Single geometry type per file
Attribute names can’t exceed 10 characters

And more…

Specialized spatial file formats

Raster data
- Digital Elevation Model (DEM)
- GeoTIFF
Map tiles
- MBTiles
- PMTiles

Specialized spatial file formats

Columnar data formats
- GeoParquet
Row oriented
- FlatGeobuf
Database services
- PostGIS
Local file databases
- GeoPackage
- duckdb (see duckspatial R package)

Historically, many tools and formats did not work together

Now, GDAL supports translation between many different formats

Read spatial data files with R

The dsn or data source name for sf::read_sf() can include:

a file name
a URL
a folder
the name and access credentials of a database
a character string with the contents of a GeoJSON file

Read spatial data files with R

You can use the query parameter of sf::read_sf() to use a SQL query to select records from a file (or select a specific geographical extent).

Many file formats can include spatial data

Non-spatial files can still be used to store Well Known Text (WKT) strings, coordinate values, or addresses (convertible to spatial data):

Excel files (xlsx or xls)
vCard or VCF (Virtual Contact File)
EXIF metadata in a JPEG, HEIC, PNG, TIFF or other media file

Reading data into R

Read data using {sf}
Read data using a package that returns a sf object
Read data and use {sf} to convert to a sf object

Writing data from R

Write data to a spatial file using {sf}
Write data a a non-spatial file using a different package

When you write spatial data to a non-spatial format, you should drop the geometry or convert the geometry to coordinates or well-known text.

Read data using `{sf}`

First load {sf}:

library(sf)

Read data using `{sf}`

{sf} can use any available drivers to read data:

drivers <- sf::st_drivers()

drivers

                         name
PCIDSK                 PCIDSK
netCDF                 netCDF
PDS4                     PDS4
VICAR                   VICAR
JP2OpenJPEG       JP2OpenJPEG
PDF                       PDF
MBTiles               MBTiles
BAG                       BAG
EEDA                     EEDA
OGCAPI                 OGCAPI
ESRI Shapefile ESRI Shapefile
MapInfo File     MapInfo File
UK .NTF               UK .NTF
LVBAG                   LVBAG
OGR_SDTS             OGR_SDTS
S57                       S57
DGN                       DGN
OGR_VRT               OGR_VRT
Memory                 Memory
CSV                       CSV
GML                       GML
GPX                       GPX
KML                       KML
GeoJSON               GeoJSON
GeoJSONSeq         GeoJSONSeq
ESRIJSON             ESRIJSON
TopoJSON             TopoJSON
OGR_GMT               OGR_GMT
GPKG                     GPKG
SQLite                 SQLite
ODBC                     ODBC
WAsP                     WAsP
PGeo                     PGeo
MSSQLSpatial     MSSQLSpatial
PostgreSQL         PostgreSQL
OpenFileGDB       OpenFileGDB
DXF                       DXF
CAD                       CAD
FlatGeobuf         FlatGeobuf
Geoconcept         Geoconcept
GeoRSS                 GeoRSS
VFK                       VFK
PGDUMP                 PGDUMP
OSM                       OSM
GPSBabel             GPSBabel
OGR_PDS               OGR_PDS
WFS                       WFS
OAPIF                   OAPIF
EDIGEO                 EDIGEO
SVG                       SVG
Idrisi                 Idrisi
XLS                       XLS
ODS                       ODS
XLSX                     XLSX
Elasticsearch   Elasticsearch
Carto                   Carto
AmigoCloud         AmigoCloud
SXF                       SXF
Selafin               Selafin
JML                       JML
PLSCENES             PLSCENES
CSW                       CSW
VDV                       VDV
MVT                       MVT
NGW                       NGW
MapML                   MapML
GTFS                     GTFS
PMTiles               PMTiles
JSONFG                 JSONFG
TIGER                   TIGER
AVCBin                 AVCBin
AVCE00                 AVCE00
HTTP                     HTTP
                                                          long_name write  copy
PCIDSK                                         PCIDSK Database File  TRUE FALSE
netCDF                                   Network Common Data Format  TRUE  TRUE
PDS4                                   NASA Planetary Data System 4  TRUE  TRUE
VICAR                                               MIPL VICAR file  TRUE  TRUE
JP2OpenJPEG              JPEG-2000 driver based on OpenJPEG library FALSE  TRUE
PDF                                                  Geospatial PDF  TRUE  TRUE
MBTiles                                                     MBTiles  TRUE  TRUE
BAG                                      Bathymetry Attributed Grid  TRUE  TRUE
EEDA                                          Earth Engine Data API FALSE FALSE
OGCAPI                                                       OGCAPI FALSE FALSE
ESRI Shapefile                                       ESRI Shapefile  TRUE FALSE
MapInfo File                                           MapInfo File  TRUE FALSE
UK .NTF                                                     UK .NTF FALSE FALSE
LVBAG                                   Kadaster LV BAG Extract 2.0 FALSE FALSE
OGR_SDTS                                                       SDTS FALSE FALSE
S57                                                  IHO S-57 (ENC)  TRUE FALSE
DGN                                                Microstation DGN  TRUE FALSE
OGR_VRT                                    VRT - Virtual Datasource FALSE FALSE
Memory                                                       Memory  TRUE FALSE
CSV                                    Comma Separated Value (.csv)  TRUE FALSE
GML                                 Geography Markup Language (GML)  TRUE FALSE
GPX                                                             GPX  TRUE FALSE
KML                                   Keyhole Markup Language (KML)  TRUE FALSE
GeoJSON                                                     GeoJSON  TRUE FALSE
GeoJSONSeq                                         GeoJSON Sequence  TRUE FALSE
ESRIJSON                                                   ESRIJSON FALSE FALSE
TopoJSON                                                   TopoJSON FALSE FALSE
OGR_GMT                                    GMT ASCII Vectors (.gmt)  TRUE FALSE
GPKG                                                     GeoPackage  TRUE  TRUE
SQLite                                          SQLite / Spatialite  TRUE FALSE
ODBC                                                                FALSE FALSE
WAsP                                               WAsP .map format  TRUE FALSE
PGeo                                      ESRI Personal GeoDatabase FALSE FALSE
MSSQLSpatial                  Microsoft SQL Server Spatial Database  TRUE FALSE
PostgreSQL                                       PostgreSQL/PostGIS  TRUE FALSE
OpenFileGDB                                            ESRI FileGDB  TRUE FALSE
DXF                                                     AutoCAD DXF  TRUE FALSE
CAD                                                  AutoCAD Driver FALSE FALSE
FlatGeobuf                                               FlatGeobuf  TRUE FALSE
Geoconcept                                               Geoconcept  TRUE FALSE
GeoRSS                                                       GeoRSS  TRUE FALSE
VFK                            Czech Cadastral Exchange Data Format FALSE FALSE
PGDUMP                                          PostgreSQL SQL dump  TRUE FALSE
OSM                                       OpenStreetMap XML and PBF FALSE FALSE
GPSBabel                                                   GPSBabel  TRUE FALSE
OGR_PDS                                Planetary Data Systems TABLE FALSE FALSE
WFS                                   OGC WFS (Web Feature Service) FALSE FALSE
OAPIF                                            OGC API - Features FALSE FALSE
EDIGEO                                French EDIGEO exchange format FALSE FALSE
SVG                                        Scalable Vector Graphics FALSE FALSE
Idrisi                                         Idrisi Vector (.vct) FALSE FALSE
XLS                                                 MS Excel format FALSE FALSE
ODS            Open Document/ LibreOffice / OpenOffice Spreadsheet   TRUE FALSE
XLSX                                 MS Office Open XML spreadsheet  TRUE FALSE
Elasticsearch                                        Elastic Search  TRUE FALSE
Carto                                                         Carto  TRUE FALSE
AmigoCloud                                               AmigoCloud  TRUE FALSE
SXF                                     Storage and eXchange Format FALSE FALSE
Selafin                                                     Selafin  TRUE FALSE
JML                                                    OpenJUMP JML  TRUE FALSE
PLSCENES                                     Planet Labs Scenes API FALSE FALSE
CSW                          OGC CSW (Catalog  Service for the Web) FALSE FALSE
VDV                             VDV-451/VDV-452/INTREST Data Format  TRUE FALSE
MVT                                             Mapbox Vector Tiles  TRUE FALSE
NGW                                                     NextGIS Web  TRUE  TRUE
MapML                                                         MapML  TRUE FALSE
GTFS                             General Transit Feed Specification FALSE FALSE
PMTiles                                              ProtoMap Tiles  TRUE FALSE
JSONFG                             OGC Features and Geometries JSON  TRUE FALSE
TIGER                                        U.S. Census TIGER/Line FALSE FALSE
AVCBin                                     Arc/Info Binary Coverage FALSE FALSE
AVCE00                                Arc/Info E00 (ASCII) Coverage FALSE FALSE
HTTP                                          HTTP Fetching Wrapper FALSE FALSE
               is_raster is_vector   vsi
PCIDSK              TRUE      TRUE  TRUE
netCDF              TRUE      TRUE FALSE
PDS4                TRUE      TRUE  TRUE
VICAR               TRUE      TRUE  TRUE
JP2OpenJPEG         TRUE      TRUE  TRUE
PDF                 TRUE      TRUE FALSE
MBTiles             TRUE      TRUE  TRUE
BAG                 TRUE      TRUE  TRUE
EEDA               FALSE      TRUE FALSE
OGCAPI              TRUE      TRUE  TRUE
ESRI Shapefile     FALSE      TRUE  TRUE
MapInfo File       FALSE      TRUE  TRUE
UK .NTF            FALSE      TRUE  TRUE
LVBAG              FALSE      TRUE  TRUE
OGR_SDTS           FALSE      TRUE  TRUE
S57                FALSE      TRUE  TRUE
DGN                FALSE      TRUE  TRUE
OGR_VRT            FALSE      TRUE  TRUE
Memory             FALSE      TRUE FALSE
CSV                FALSE      TRUE  TRUE
GML                FALSE      TRUE  TRUE
GPX                FALSE      TRUE  TRUE
KML                FALSE      TRUE  TRUE
GeoJSON            FALSE      TRUE  TRUE
GeoJSONSeq         FALSE      TRUE  TRUE
ESRIJSON           FALSE      TRUE  TRUE
TopoJSON           FALSE      TRUE  TRUE
OGR_GMT            FALSE      TRUE  TRUE
GPKG                TRUE      TRUE  TRUE
SQLite             FALSE      TRUE  TRUE
ODBC               FALSE      TRUE FALSE
WAsP               FALSE      TRUE  TRUE
PGeo               FALSE      TRUE FALSE
MSSQLSpatial       FALSE      TRUE FALSE
PostgreSQL         FALSE      TRUE FALSE
OpenFileGDB         TRUE      TRUE  TRUE
DXF                FALSE      TRUE  TRUE
CAD                 TRUE      TRUE  TRUE
FlatGeobuf         FALSE      TRUE  TRUE
Geoconcept         FALSE      TRUE  TRUE
GeoRSS             FALSE      TRUE  TRUE
VFK                FALSE      TRUE FALSE
PGDUMP             FALSE      TRUE  TRUE
OSM                FALSE      TRUE  TRUE
GPSBabel           FALSE      TRUE FALSE
OGR_PDS            FALSE      TRUE  TRUE
WFS                FALSE      TRUE  TRUE
OAPIF              FALSE      TRUE FALSE
EDIGEO             FALSE      TRUE  TRUE
SVG                FALSE      TRUE  TRUE
Idrisi             FALSE      TRUE  TRUE
XLS                FALSE      TRUE FALSE
ODS                FALSE      TRUE  TRUE
XLSX               FALSE      TRUE  TRUE
Elasticsearch      FALSE      TRUE FALSE
Carto              FALSE      TRUE FALSE
AmigoCloud         FALSE      TRUE FALSE
SXF                FALSE      TRUE  TRUE
Selafin            FALSE      TRUE  TRUE
JML                FALSE      TRUE  TRUE
PLSCENES            TRUE      TRUE FALSE
CSW                FALSE      TRUE FALSE
VDV                FALSE      TRUE  TRUE
MVT                FALSE      TRUE  TRUE
NGW                 TRUE      TRUE FALSE
MapML              FALSE      TRUE  TRUE
GTFS               FALSE      TRUE  TRUE
PMTiles            FALSE      TRUE  TRUE
JSONFG             FALSE      TRUE  TRUE
TIGER              FALSE      TRUE  TRUE
AVCBin             FALSE      TRUE  TRUE
AVCE00             FALSE      TRUE  TRUE
HTTP                TRUE      TRUE FALSE

Read data from a URL using `{sf}`

read_sf() works well with a direct link to a spatial data file.

Read a GeoJSON file:

# GeoJSON (geojson)
read_sf(
  "https://github.com/ropensci/geojsonio/raw/main/inst/examples/california.geojson"
)

Simple feature collection with 1 feature and 11 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: -124.4096 ymin: 32.53416 xmax: -114.1315 ymax: 42.00952
Geodetic CRS:  WGS 84
# A tibble: 1 × 12
  name       abbreviation capital    city   population   area waterarea landarea
  <chr>      <chr>        <chr>      <chr>       <int>  <int>     <int>    <int>
1 California CA           Sacramento Los A…   38332521 423968     20502   403466
# ℹ 4 more variables: houseseats <int>, statehood <date>, group <chr>,
#   geometry <MULTIPOLYGON [°]>

Read a KML file:

# Keyhole Markup Language (kml)
read_sf(
  "https://github.com/ropensci/geojsonio/raw/main/inst/examples/poa_annua.kml"
)

Simple feature collection with 161 features and 2 fields
Geometry type: POINT
Dimension:     XY
Bounding box:  xmin: -167.15 ymin: 0 xmax: 0 ymax: 69.3333
Geodetic CRS:  WGS 84
# A tibble: 161 × 3
   Name                                    Description             geometry
   <chr>                                   <chr>                <POINT [°]>
 1 "Patterson's bluegrass, Patterson blue… "<table wi… (-113.9553 43.39448)
 2 "Patterson's bluegrass, Patterson blue… "<table wi…  (-114.1191 46.0793)
 3 "Patterson's bluegrass, Patterson blue… "<table wi…  (-114.1191 46.0793)
 4 "Patterson's bluegrass, Patterson blue… "<table wi…  (-114.1191 46.0793)
 5 "Patterson's bluegrass, Patterson blue… "<table wi… (-109.5936 44.49239)
 6 "Patterson's bluegrass, Patterson blue… "<table wi…  (-114.1191 46.0793)
 7 "Patterson's bluegrass, Patterson blue… "<table wi…   (-109.817 40.7667)
 8 "Patterson's bluegrass, Patterson blue… "<table wi…            (-105 39)
 9 "Patterson's bluegrass, Patterson blue… "<table wi… (-106.1376 39.62102)
10 "Patterson's bluegrass, Patterson blue… "<table wi…   (-111.183 40.6167)
# ℹ 151 more rows

Read a GeoPackage file:

# GeoPackage (gpkg)
read_sf("https://github.com/r-spatial/sf/raw/main/inst/gpkg/nc.gpkg")

Simple feature collection with 100 features and 14 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: -84.32385 ymin: 33.88199 xmax: -75.45698 ymax: 36.58965
Geodetic CRS:  NAD27
# A tibble: 100 × 15
    AREA PERIMETER CNTY_ CNTY_ID NAME  FIPS  FIPSNO CRESS_ID BIR74 SID74 NWBIR74
   <dbl>     <dbl> <dbl>   <dbl> <chr> <chr>  <dbl>    <int> <dbl> <dbl>   <dbl>
 1 0.114      1.44  1825    1825 Ashe  37009  37009        5  1091     1      10
 2 0.061      1.23  1827    1827 Alle… 37005  37005        3   487     0      10
 3 0.143      1.63  1828    1828 Surry 37171  37171       86  3188     5     208
 4 0.07       2.97  1831    1831 Curr… 37053  37053       27   508     1     123
 5 0.153      2.21  1832    1832 Nort… 37131  37131       66  1421     9    1066
 6 0.097      1.67  1833    1833 Hert… 37091  37091       46  1452     7     954
 7 0.062      1.55  1834    1834 Camd… 37029  37029       15   286     0     115
 8 0.091      1.28  1835    1835 Gates 37073  37073       37   420     0     254
 9 0.118      1.42  1836    1836 Warr… 37185  37185       93   968     4     748
10 0.124      1.43  1837    1837 Stok… 37169  37169       85  1612     1     160
# ℹ 90 more rows
# ℹ 4 more variables: BIR79 <dbl>, SID79 <dbl>, NWBIR79 <dbl>,
#   geom <MULTIPOLYGON [°]>

Even non-spatial data works if the data includes a well-known text column:

# Comma-separated values (csv)
read_sf("https://github.com/r-spatial/sf/raw/main/inst/csv/pt.csv")

Simple feature collection with 3 features and 4 fields
Geometry type: POINT
Dimension:     XY
Bounding box:  xmin: 0 ymin: 0 xmax: 44 ymax: 55
CRS:           NA
# A tibble: 3 × 5
  Id    Int64        Int32 WKT          geometry
  <chr> <chr>        <chr> <chr>         <POINT>
1 1     "0"          "33"  POINT(0 1)      (0 1)
2 2     "4294967297" "44"  POINT(44 55)  (44 55)
3 3     ""           ""    POINT(0 0)      (0 0)

If the non-spatial data does not include a well-known text column, read_sf() still returns a data frame:

read_sf(
  "https://github.com/elipousson/marylandedu/raw/main/data-raw/extdata/Enrollment_By_Grade_2023.csv"
)

# A tibble: 9,766 × 8
   Year  LEA   `LEA Name` School `School Name`         Grade    `Enrolled Count`
   <chr> <chr> <chr>      <chr>  <chr>                 <chr>    <chr>           
 1 2023  01    Allegany   0301   Flintstone Elementary Prekind… 19              
 2 2023  01    Allegany   0301   Flintstone Elementary Kinderg… 42              
 3 2023  01    Allegany   0301   Flintstone Elementary Grade 1  33              
 4 2023  01    Allegany   0301   Flintstone Elementary Grade 2  29              
 5 2023  01    Allegany   0301   Flintstone Elementary Grade 3  26              
 6 2023  01    Allegany   0301   Flintstone Elementary Grade 4  31              
 7 2023  01    Allegany   0301   Flintstone Elementary Grade 5  36              
 8 2023  01    Allegany   0301   Flintstone Elementary All Gra… 216             
 9 2023  01    Allegany   0401   South Penn Elementary Prekind… 77              
10 2023  01    Allegany   0401   South Penn Elementary Kinderg… 79              
# ℹ 9,756 more rows
# ℹ 1 more variable: `Create Date` <chr>

A multifile format like a shapefile does not work as a URL:

# Shapefile (shp)
read_sf("https://github.com/r-spatial/sf/raw/main/inst/shape/nc.shp")

Error:
! Cannot open "https://github.com/r-spatial/sf/raw/main/inst/shape/nc.shp"; The file doesn't seem to exist.

But it does work if you provide a local file path (system.file() returns a file path in this case):

nc <- read_sf(system.file("shape/nc.shp", package = "sf"))

Practice: Use a spatial query in `wkt_filter`

wkt_filter <- st_as_text(nc[1, ]$geometry)

read_sf(
  system.file("shape/nc.shp", package = "sf"),
  wkt_filter = wkt_filter
)

Simple feature collection with 4 features and 14 fields
Geometry type: POLYGON
Dimension:     XY
Bounding box:  xmin: -81.91116 ymin: 35.98933 xmax: -80.87086 ymax: 36.58965
Geodetic CRS:  NAD27
# A tibble: 4 × 15
   AREA PERIMETER CNTY_ CNTY_ID NAME   FIPS  FIPSNO CRESS_ID BIR74 SID74 NWBIR74
  <dbl>     <dbl> <dbl>   <dbl> <chr>  <chr>  <dbl>    <int> <dbl> <dbl>   <dbl>
1 0.114      1.44  1825    1825 Ashe   37009  37009        5  1091     1      10
2 0.061      1.23  1827    1827 Alleg… 37005  37005        3   487     0      10
3 0.199      1.98  1874    1874 Wilkes 37193  37193       97  3146     4     200
4 0.081      1.29  1880    1880 Watau… 37189  37189       95  1323     1      17
# ℹ 4 more variables: BIR79 <dbl>, SID79 <dbl>, NWBIR79 <dbl>,
#   geometry <POLYGON [°]>

Working with web services

Examples of common web services for working with spatial data include:

Spatial data sharing and discovery
Spatial data visualization
Routing
Geocoding

Considerations when working with web services

Using a web service often requires:

Creating an account for authentication and billing
Dedicating funding for usage
Understanding data privacy implications

Example: ArcGIS Feature Services with `read_sf`

An ArcGIS FeatureServer URL does not work:

read_sf(
  "https://services.arcgis.com/njFNhDsUCentVYJW/arcgis/rest/services/MDOT_SHA_County_Boundaries/FeatureServer/0"
)

Error:
! Cannot open "https://services.arcgis.com/njFNhDsUCentVYJW/arcgis/rest/services/MDOT_SHA_County_Boundaries/FeatureServer/0"; The file doesn't seem to exist.

But a FeatureServer URL that includes the output format does work:

read_sf(
  "https://services.arcgis.com/njFNhDsUCentVYJW/arcgis/rest/services/MDOT_SHA_County_Boundaries/FeatureServer/0/query?outFields=*&where=1%3D1&f=geojson"
)

Simple feature collection with 24 features and 18 fields
Geometry type: POLYGON
Dimension:     XY
Bounding box:  xmin: -79.48731 ymin: 37.88654 xmax: -75.0492 ymax: 39.72288
Geodetic CRS:  WGS 84
# A tibble: 24 × 19
   OBJECTID COUNTY_NAME     COUNTY_INTEGER COUNTY_ABBREVIATION COUNTY_DIGITCAPS
      <int> <chr>                    <int> <chr>                          <int>
 1        1 Prince George's             16 PG                                16
 2        2 Cecil                        7 CE                                 7
 3        3 Howard                      13 HO                                13
 4        4 Talbot                      20 TA                                20
 5        5 Garrett                     11 GA                                11
 6        6 Baltimore City              24 BC                                24
 7        7 Carroll                      6 CL                                 6
 8        8 Washington                  21 WA                                21
 9        9 Worcester                   23 WO                                23
10       10 Frederick                   10 FR                                10
# ℹ 14 more rows
# ℹ 14 more variables: COUNTY_2DIGITCAPS <chr>, DISTRICT_NAME <chr>,
#   DISTRICT_NUMBER <int>, COUNTY_FIP <int>, COUNTYNUM <int>, TSD_ID <int>,
#   COUNTY <chr>, DISTRICT <int>, Shape__Area <dbl>, Shape__Length <dbl>,
#   COUNTY_ALLCAPS <chr>, Region <chr>, GlobalID <chr>, geometry <POLYGON [°]>

Example: ArcGIS Feature Services with `arc_select`

For ArcGIS Feature Services, the {arcgislayers} package is the recommended way to access data:

library(arcgislayers)

feature_url <- arc_open(
  "https://services.arcgis.com/njFNhDsUCentVYJW/arcgis/rest/services/MDOT_SHA_County_Boundaries/FeatureServer/0"
)

arc_select(feature_url)

Simple feature collection with 24 features and 18 fields
Geometry type: POLYGON
Dimension:     XY
Bounding box:  xmin: 607669.3 ymin: 80997.33 xmax: 1870976 ymax: 757701.4
Projected CRS: NAD83(HARN) / Maryland (ftUS)
First 10 features:
   OBJECTID     COUNTY_NAME COUNTY_INTEGER COUNTY_ABBREVIATION COUNTY_DIGITCAPS
1         1 Prince George's             16                  PG               16
2         2           Cecil              7                  CE                7
3         3          Howard             13                  HO               13
4         4          Talbot             20                  TA               20
5         5         Garrett             11                  GA               11
6         6  Baltimore City             24                  BC               24
7         7         Carroll              6                  CL                6
8         8      Washington             21                  WA               21
9         9       Worcester             23                  WO               23
10       10       Frederick             10                  FR               10
   COUNTY_2DIGITCAPS DISTRICT_NAME DISTRICT_NUMBER COUNTY_FIP COUNTYNUM TSD_ID
1                 16    District 3               3         33        16      3
2                 07    District 2               2         15         7     21
3                 13    District 7               7         27        13     10
4                 20    District 2               2         41        20     18
5                 11    District 6               6         23        11      4
6                 24   No District               0        510        24     23
7                 06    District 7               7         13         6      1
8                 21    District 6               6         43        21   1060
9                 23    District 1               1         47        23      2
10                10    District 7               7         21        10     24
            COUNTY DISTRICT Shape__Area Shape__Length  COUNTY_ALLCAPS Region
1  Prince George's        3 13902243040      720691.0 PRINCE GEORGE'S   <NA>
2            Cecil        2 11643675836      494670.7           CECIL   <NA>
3           Howard        7  7056288549      471922.1          HOWARD   <NA>
4           Talbot        2 13193323636      657963.0          TALBOT   <NA>
5          Garrett        6 18339518744      680782.9         GARRETT   <NA>
6   Baltimore City        0  2562810672      207145.0  BALTIMORE CITY   <NA>
7          Carroll        7 12623420879      651107.6         CARROLL   <NA>
8       Washington        6 13031238277      948164.4      WASHINGTON   <NA>
9        Worcester        1 16297527948      694972.5       WORCESTER   <NA>
10       Frederick        7 18613257293      683407.8       FREDERICK   <NA>
                               GlobalID                       geometry
1  8dec0b24-8a66-47c5-97ec-aa0ef22dc80a POLYGON ((1313442 476110.4,...
2  4ea4cc1d-607c-41cd-b35c-6eae0d049b50 POLYGON ((1660741 628728.5,...
3  0241b589-d266-4eb6-8403-a0d5a6660b7d POLYGON ((1391417 567919.8,...
4  f14b3860-0b31-41d6-a0f9-8f16468e10d4 POLYGON ((1611348 457521.9,...
5  247e3d29-1b5c-423b-a793-5c694d42cf34 POLYGON ((769889 754580.7, ...
6  89c9198d-40cb-4f46-8b08-6b62b0b29885 POLYGON ((1445534 565803.6,...
7  c8aa72df-586c-4ba2-97de-b86a6e486b0d POLYGON ((1371781 748182.4,...
8  eea627a4-f12d-453a-a9dc-8dbb3fcdf826 POLYGON ((936016.4 751530.6...
9  70d147e2-cc68-470b-bc8c-41bdcea19451 POLYGON ((1870321 284172.9,...
10 ae254bf4-2ea3-4c79-ae24-a83a993c7b20 POLYGON ((1213130 747989.9,...

Example: ArcGIS Portal Items with `arc_open`

{arcgislayers} also has support for getting data on “portal items” like web maps and applications:

app <- arc_open(
  "https://maryland.maps.arcgis.com/apps/webappviewer/index.html?id=e4f351766e614b7195525b49b8be112e"
)

app

<PortalItem<Web Mapping Application>>
id: e4f351766e614b7195525b49b8be112e
title: MDOT SHA Traffic Volume County Maps
owner: marshall.stevenson1

Practice: Write a SQL expression to query a FeatureLayer

Use the format “{Field_name} {Operator} {Value or String}” to build a query and filter data before loading it to your computer.

Check out the SQL reference for ArcGIS query expressions for more information.

url <- "https://services1.arcgis.com/UWYHeuuJISiGmgXx/arcgis/rest/services/311_Customer_Service_Requests_2024/FeatureServer/0"

layer <- arc_open(url)

arc_select(
  layer,
  where = "CloseDate > date '2024-01-01' AND CloseDate < date '2024-01-06'",
  n_max = 10
)

Simple feature collection with 10 features and 23 fields
Geometry type: POINT
Dimension:     XY
Bounding box:  xmin: -76.68608 ymin: 39.30426 xmax: -76.63588 ymax: 39.34304
Geodetic CRS:  WGS 84
      RowID         SRRecordID ServiceRequestNum
1  10054845 5004w00002sWMk0AAG       24-00000007
2  10054846 5004w00002sWMkQAAW       24-00000008
3  10054850 5004w00002sWMpeAAG       24-00000001
4  10054852 5004w00002sWMxJAAW       24-00000021
5  10054864 5004w00002sWN2cAAG       24-00000005
6  10054865 5004w00002sWN2mAAG       24-00000006
7  10054866 5004w00002sWN2NAAW       24-00000002
8  10054867 5004w00002sWN2rAAG       24-00000009
9  10054869 5004w00002sWN2wAAG       24-00000011
10 10054872 5004w00002sWN3BAAW       24-00000013
                                  SRType MethodReceived         CreatedDate
1  TRS-48 Hour Parking/Abandoned Vehicle            API 2023-12-31 19:11:10
2                    SW-Graffiti Removal            API 2023-12-31 19:12:21
3                ECC-Information Request          Phone 2023-12-31 19:01:11
4                ECC-Information Request          Phone 2023-12-31 19:30:43
5                ECC-Information Request          Phone 2023-12-31 19:08:13
6                        SW-Dirty Street            API 2023-12-31 19:08:56
7                ECC-Information Request          Phone 2023-12-31 19:03:46
8                ECC-Information Request          Phone 2023-12-31 19:12:23
9                ECC-Information Request          Phone 2023-12-31 19:15:51
10               ECC-Information Request          Phone 2023-12-31 19:19:49
   SRStatus          StatusDate             DueDate           CloseDate
1    Closed 2024-01-05 10:58:07 2024-01-07 19:11:09 2024-01-05 10:58:07
2    Closed 2024-01-03 09:45:00 2024-01-14 19:12:21 2024-01-03 09:45:00
3    Closed 2023-12-31 19:02:00 2023-12-31 14:01:11 2023-12-31 19:02:00
4    Closed 2023-12-31 19:31:30 2023-12-31 14:30:43 2023-12-31 19:31:30
5    Closed 2023-12-31 19:09:08 2023-12-31 14:08:13 2023-12-31 19:09:08
6    Closed 2024-01-03 09:31:00 2024-01-09 19:08:56 2024-01-03 09:31:00
7    Closed 2023-12-31 19:04:01 2023-12-31 14:03:46 2023-12-31 19:04:00
8    Closed 2023-12-31 19:13:03 2023-12-31 14:12:23 2023-12-31 19:13:03
9    Closed 2023-12-31 19:16:05 2023-12-31 14:15:51 2023-12-31 19:16:05
10   Closed 2023-12-31 19:20:04 2023-12-31 14:19:49 2023-12-31 19:20:04
                                               Agency     LastActivity
1  Transportation                                     Service Response
2  Solid Waste                                        Service Response
3  ECC                                                            <NA>
4  ECC                                                            <NA>
5  ECC                                                            <NA>
6  Solid Waste                                        Service Response
7  ECC                                                            <NA>
8  ECC                                                            <NA>
9  ECC                                                            <NA>
10 ECC                                                            <NA>
      LastActivityDate                                             Outcome
1  2024-01-05 10:58:07 SR closed by agency-specific work management system
2  2024-01-03 09:45:00                         Work could not be completed
3                 <NA>                                                <NA>
4                 <NA>                                                <NA>
5                 <NA>                                                <NA>
6  2024-01-03 09:31:00      SR assessed and no cause for action determined
7                 <NA>                                                <NA>
8                 <NA>                                                <NA>
9                 <NA>                                                <NA>
10                <NA>                                                <NA>
                                        Address ZipCode Neighborhood
1      5242 FAIRLAWN AVE, Baltimore City, 21215   21215     Woodmere
2      1909 N FULTON AVE, Baltimore City, 21217   21217   Penn North
3                                          <NA>    <NA>         <NA>
4                                          <NA>    <NA>         <NA>
5                                          <NA>    <NA>         <NA>
6  1800 PENNSYLVANIA AVE, Baltimore City, 21217   21217        Upton
7                                          <NA>    <NA>         <NA>
8                                          <NA>    <NA>         <NA>
9                                          <NA>    <NA>         <NA>
10                                         <NA>    <NA>         <NA>
   CouncilDistrict PoliceDistrict PolicePost           Latitude
1                6   Northwestern        633 39.343041765917619
2                7        Western        731 39.310466431895868
3             <NA>           <NA>       <NA>               <NA>
4             <NA>           <NA>       <NA>               <NA>
5             <NA>           <NA>       <NA>               <NA>
6               11        Western        713 39.304256585322513
7             <NA>           <NA>       <NA>               <NA>
8             <NA>           <NA>       <NA>               <NA>
9             <NA>           <NA>       <NA>               <NA>
10            <NA>           <NA>       <NA>               <NA>
             Longitude                               GeoLocation
1  -76.686078294484034 (39.343041765917619, -76.686078294484034)
2  -76.646214120936790 (39.310466431895868, -76.646214120936790)
3                 <NA>                                      <NA>
4                 <NA>                                      <NA>
5                 <NA>                                      <NA>
6  -76.635878307194716 (39.304256585322513, -76.635878307194716)
7                 <NA>                                      <NA>
8                 <NA>                                      <NA>
9                 <NA>                                      <NA>
10                <NA>                                      <NA>
                     geometry
1  POINT (-76.68608 39.34304)
2  POINT (-76.64621 39.31047)
3                 POINT EMPTY
4                 POINT EMPTY
5                 POINT EMPTY
6  POINT (-76.63588 39.30426)
7                 POINT EMPTY
8                 POINT EMPTY
9                 POINT EMPTY
10                POINT EMPTY

Example: Mapbox API

mapboxapi::mb_isochrone() allows access to the Mapbox Isochrone API:

library(mapboxapi)

Error in `library()`:
! there is no package called 'mapboxapi'

walk_5min <- mb_isochrone(
  "100 Holliday St, Baltimore, MD 21202",
  profile = "walking",
  time = 5
)

Error in `mb_isochrone()`:
! could not find function "mb_isochrone"

walk_5min

Error:
! object 'walk_5min' not found

Example: Mapbox API

mapboxapi::addMapboxTiles() provides easy access to the Mapbox Static Tiles API:

library(leaflet)

leaflet(walk_5min) %>%
  addMapboxTiles(style_id = "streets-v11", username = "mapbox") %>%
  addPolygons()

Error in `addMapboxTiles()`:
! could not find function "addMapboxTiles"

mapgl::mapboxgl() is now a recommended alternative to addMapboxTiles().

Storing and organizing spatial data takes planning

Planning for the structure and organization of your data can be supported using a variety of tools:

Diagram drawing applications (e.g. draw.io)
Entity-relationship diagram tools (e.g. drawDB)
Low-code, visual database applications (e.g. Airtable)

Reading and writing spatial data

Overview

Spatial data is everywhere

Spatial data is everywhere

Spatial data is everywhere

Creating spatial data is “easy”

What can we do with it all?

Standards and specifications

Standards and specifications

Common spatial file formats

Common spatial file formats

Why are shapefiles bad?

Specialized spatial file formats

Specialized spatial file formats

Historically, many tools and formats did not work together

Now, GDAL supports translation between many different formats

Read spatial data files with R

Read spatial data files with R

Many file formats can include spatial data

Reading data into R

Writing data from R

Read data using {sf}

Read data using {sf}

Read data from a URL using {sf}

Practice: Use a spatial query in wkt_filter

Working with web services

Considerations when working with web services

Example: ArcGIS Feature Services with read_sf

Example: ArcGIS Feature Services with arc_select

Example: ArcGIS Portal Items with arc_open

Practice: Write a SQL expression to query a FeatureLayer

Example: Mapbox API

Example: Mapbox API

Storing and organizing spatial data takes planning

Read data using `{sf}`

Read data using `{sf}`

Read data from a URL using `{sf}`

Practice: Use a spatial query in `wkt_filter`

Example: ArcGIS Feature Services with `read_sf`

Example: ArcGIS Feature Services with `arc_select`

Example: ArcGIS Portal Items with `arc_open`