Problems and challenges in spatial analysis

Unfortunately, integrating geospatial data into your organization’s decision-making is not

without its obstacles. Some are common to other data integration processes. Others are
unique to geospatial data because of what it describes and how it behaves.

1. Data standardization:
Many individuals working in data science and geographic information systems (GIS) often
find themselves dedicating up to 90% of their time to data cleaning tasks before they can
begin analysis. This significant time investment stems largely from the absence of
standardized practices. For instance, timestamps may originate from different time zones, or
measurements might be recorded using varying units, sometimes with no straightforward
conversion between them (such as metric versus imperial units). Furthermore, the
effectiveness of a standard can be hindered by its adoption rate. Various obstacles may
impede widespread adoption. For instance, creators of a standard may require fees, demand
data sharing obligations, or impose other requirements that deter individuals and
organizations from embracing the standard. It's important to note that a standard doesn't need
to cater perfectly to all scenarios; rather, it should suffice for a critical mass of people or
organizations to agree upon it and derive value from its implementation.

How to solve this problem:

A robust standard should enable your datasets to be comprehensible within the context of a
wide range of other datasets. Achieving this requires the ability to identify data points
according to a set of guidelines, often encapsulated in the "S.I.M.P.L.E." formula:

- Storable: Data point IDs should be capable of being stored in locations that do not
necessitate internet access.
- Immutable: Data point IDs should remain unchanged over time, barring extreme
circumstances.
- Meticulous: Data points should possess unique identifiability across all systems in which
they are present.
- Portable: Standardized IDs should facilitate seamless transitions of data points from one
storage system or dataset to another.
- Low-cost: Utilizing the standard should incur minimal expenses, potentially even being free
for data transactions.

1
- Established: The standard must encompass nearly all conceivable data points to which it
could be applied.

2. Choice of Interpolation Method:

Spatial interpolation refers to the process of estimating unknown attribute values at
unsampled locations within a geographic area based on known values at sampled locations.
While spatial interpolation is a crucial aspect of spatial analysis, it comes with its own set of
challenges:

a) Data Quality and Distribution: The reliability of interpolated values heavily relies on the
quality and distribution of the sampled data points. Sparse or unevenly distributed data can
lead to inaccurate interpolation results, particularly in areas with limited data coverage.

b) Uncertainty and Error Assessment: Interpolated values inherently contain uncertainty,

which arises from factors such as measurement error, sampling bias, and the interpolation
method itself. Assessing the reliability and accuracy of interpolated values is challenging and
often requires statistical techniques to quantify uncertainty and error margins.

c) Boundary Effects: Interpolation methods may produce biased results near the boundaries
of the study area, especially when extrapolating beyond the extent of the sampled data.
Boundary effects can introduce inaccuracies and distortions in interpolated surfaces, affecting
the overall reliability of the analysis.

d) Scale and Resolution: The choice of interpolation method and the resolution of the input
data can significantly impact the spatial patterns and variability captured in the interpolated
surface. Balancing computational efficiency with the desired level of detail and accuracy is
essential but can be challenging, particularly when working with large datasets or fine-scale
spatial analyses.

e) Subjectivity in Method Selection: Determining the most suitable interpolation method

often involves a degree of subjectivity and expert judgment. Different methods may yield
divergent results, and the selection process may lack clear guidelines, leading to uncertainty
regarding the appropriateness of the chosen method.

How to solve this problem:

Addressing these challenges requires careful consideration of the specific characteristics of
the spatial data, thorough validation of interpolation results, and transparency in reporting the

2
associated uncertainties. Additionally, incorporating multiple interpolation methods or
exploring spatial modelling approaches can help mitigate some of the limitations inherent in
spatial interpolation.

3. Data quality
A lot of bad data exists. Most of it is caused by a lack of expertise in how to collect and
process it, or just simple human error. As we’ve already discussed, lack of standardization
plays a large part in this, as it can cause analysts to miss critical details. Other inaccuracies in
geocoding and digitizing physical places and features can cause a cascade of inconsistencies
in their geographic representation. These make it difficult, if not impossible, to accurately
measure foot traffic and other variables surrounding a business or other point of interest.

Open-source geospatial data is great because everyone can check it for mistakes and
omissions at least in theory. In reality, users should still be careful to vet open-source data
and make sure it is correct and suitable to their needs. The problem is that this process is
expensive and time-consuming, so companies will often skip it especially when they’re on a
tight deadline and need insights quickly. But the consequences of making important decisions
with inaccurate data can be even more costly.

How to solve this problem:

Take four steps to check data before using it. First, make sure it comes from reliable sources.
Second, evaluate what it’s capable of, including any gaps it may leave and any assumptions
you might make about it. Third, determine how much work it will take to get the data ready
for use. Finally, based on what you know the data can (and can’t) do, draw up a plan for what
specific function(s) it will serve in your operations.

If that sounds like a lot to go through, consider cutting down on some of the manual labor by
investing in Safe Graph’s datasets. They’re checked for accuracy and cleaned every month by
Safe Graph’s expert data technicians, so they’re among the most up-to-date and immediately
usable geospatial data sets on the market.

In summary, if you’re going to use geospatial data, first make sure you have the right people
and infrastructure to work with it properly. Then, make sure the actual data you’re using is as
accurate, standardized, and as relevant to your organization’s needs as possible. If you’d like
further help, get in touch with Safe Graph. We’re experts in managing geospatial data –
because it’s all we do.

3
4. Address standardization
Addresses indeed pose significant challenges for data standardization due to their diverse
elements and potential variations. Street names, building unit numbers, cities, regions,
countries, and mailing codes can be arranged differently in databases or might be missing
altogether. This variability makes it difficult for computer programs or algorithms to
determine if multiple addresses correspond to the same location.

Furthermore, inconsistencies such as misspellings, typos, punctuation variations, and

differing abbreviations or acronyms compound the issue. For instance, does the data
processing platform recognize that "US," "USA," "U.S.A.," "the (United) States," and
"America" all denote the same country? Can it distinguish whether "St." stands for "street" or
"saint," and in which contexts each interpretation applies?

How to solve this problem:

Addressing these challenges necessitates storing address data in a more efficient and
standardized manner. This is where Place key comes in: it offers a free, open, and concise
standard for representing location information. Place key generates a unique "what @ where"
string of encoded characters, first identifying a location's address and any specific points of
interest. It then defines the geographic area occupied by that location using a hexagon
cantered on its precise latitude and longitude coordinates.

5. Lack of institutional knowledge: Traditionally, geospatial data and geographic

information systems (GIS) have stood apart from fields like data science or engineering,
forming a distinct domain. Consequently, only a small segment of individuals in these latter
fields (approximately 5%) possess the necessary expertise to manipulate geospatial data.
Given its unique behaviour compared to tabular data, many organizations encounter
challenges incorporating it into their workflows due to a deficiency in relevant skills.

Closing this skills gap presents its own set of challenges, not solely due to the limited
availability of qualified professionals but also because companies must ensure they recruit
individuals possessing the requisite skill sets and experience. This often elongates the
recruitment process from crafting job postings to conducting interviews and technical
assessments beyond typical durations, which can hinder the progress of ongoing projects
within the organization. Consequently, hiring managers often face considerable pressure to
expedite the hiring process, prioritizing speed over finding the most suitable candidate for the
specific role.

4
How to solve this problem:
To address this issue, start by leveraging existing connections within the company's network.
Additionally, consider innovative approaches such as hosting webinars, hackathons, or
meetups, participating in relevant conferences, or enlisting the services of specialized
recruitment agencies to attract individuals with specialized knowledge of geospatial data.

Ideally, the desired candidate should possess robust programming skills, a background in
statistics, proficiency in developing data products, creating visualizations, establishing
workflows, and implementing pipelining routines. Furthermore, they should be familiar with
machine learning, distributed computing, and, naturally, GIS software.

6. File size/processing times

Geospatial analytics, like any data science analysis, necessitates suitable systems and
infrastructure. While you may not require radically different tools compared to other data
analyses, basic solutions like Excel and SQL-based OCDB systems might not suffice when
dealing with numerous datasets or aiming to scale up in the future.

Determining the extent of data preprocessing or optimization during analysis is crucial,

balancing cost-efficiency against the flexibility to address unique queries effectively.
Moreover, communicating these decisions with stakeholders is essential to manage
expectations regarding the speed and completeness of responses, based on data processing
capabilities.

How to solve this problem:

To address these challenges, data experts recommend leveraging cloud-based data platforms
such as S3. Although these platforms may require more time and expertise to operate and
manage, they offer superior processing capabilities and scalability over time. Additionally,
they allow for the development of custom components for the technology stack as required.
Your system should ideally include components like a data lake, a robust data storage system,
a processing platform, a task scheduler, and a tool for pipeline creation.

References:
https://www.safegraph.com/guides/geospatial-data-integration-challenges
https://www.researchgate.net/publication/220649648_GIS_and_Spatial_Analytical_Problems
https://2023.sigmod.org/tutorials/tutorial6.pdf
https://www.slideserve.com/sienna/spatial-analysis