Update on Imagery and GIS data

Update by Carl Zimmerman, PhD, CT GIS Office 

Flight blocks

This is an update from the CT GIS Office on the production, review, and publication of the imagery and GIS data collected during the spring 2023 flight. The production is divided into four geographic blocks: Block 1 (northwest), Block 2 (southwest), Block 3 (central), and Block 4 (eastern).

The four steps in the data production process are:

  1. data is collected and processed by the Vendor;
  2. it is reviewed by the GIS Office;
  3. errors and issues are sent back to the Vendor who assesses and corrects them, and then
  4. the final product is delivered to the GIS Office for publication and access by CT ECO and/or the GIS Office. 

Quality Assessment/Quality Control (QA/QC) Process

There are multiple levels in the QA/QC process. First is an initial data assessment and quality review by the Vendor (part of step 1, above) which includes numerous validation steps and a review by their subcontractor. Once received, a second QA/QC review is performed by the GIS Office and participating stakeholders (part of step 2, above) where the team assesses different areas of the different datasets in order to identify any issues or errors within the massive data sets. Currently, we are in the most intense part of the QA/QC process and are evaluating some issues that have been identified by the Vendor and our own review. We are working to determine and negotiate a mutually acceptable set of solutions which impacts the speed of delivery to our users. 

Status of the Imagery and GIS data sets 

True Color Ortho Imagery (RGB) 

The initial delivery of Blocks 1, 2, and 4 to the GISO has occurred.

  • Block 1 errors have been corrected by the Vendor (step 3, above) and should be arriving at the GISO soon (step 4)!
  • Block 2 issues are being reviewed by the GISO now (step 2) and will be sent back to the Vendor for correction this week or next.
  • Block 3 is coming the next two weeks (step 1).
  • Block 4 issues are being reviewed (step 2) and will be sent back to the Vendor next week (step 3).

We expect the full and final imagery dataset to be available in July. In the meantime, CT ECO will host imagery blocks as soon as they are delivered, although they should be considered “beta” versions.

Near Infrared Imagery (NIR) 

The near-infrared band is collected with the true color imagery and is usually treated as one. Unfortunately, a technical issue has been identified near the shoreline for Block 2 where the near-infrared data is “bleeding” into the other areas and is impacting the usability of the data. The delivery (step 1) for the coastal parts of Blocks 2, 3 and 4 is on hold until this issue has been solved and the imagery has been resubmitted by the Vendor to the GISO. Updates to come. 

Lidar-Derived DEM and Contours 

The digital elevation model (DEM) is a 2 foot pixel representation of the bare earth elevation. The DEM creation is progressing well and is expected to be finished in the late summer. The 1-foot contours data set is derived from the DEM and will be delivered after the DEM in early fall.  

2D and 3D Building Footprints 

The new 2D building footprint layer will include about 1.9 million buildings across Connecticut. The initial review of Blocks 1 and 2 is mostly finished but some quality issues were identified, and iterative improvements are being performed. Blocks 3 and 4 will not be delivered until the quality issue is resolved. This data set will be available end of summer or early fall. The 3D Building Footprints will be completed last to ensure that all iterative corrections are accounted for. Expect the 3D Building Footprints in the fall.   

Classified Lidar Points

The classified lidar is meeting specifications for capture density, overlap, and classification quality. This data set will be delivered in the fall after the other data sets are completed and produced. 

Lakes, Streams, and Shoreline Polygons and Polylines 

These data sets are used for creation of other datasets, particularly the elevation. They will be delivered in the summer. 

CT Imagery Delivery Update

Written by Carl Zimmerman, PhD, GIS Coordinator, CT GIS Office (email: carl.zimmerman@ct.gov

The first wave of Imagery, Elevation, and GIS data are now being delivered to the GIS Office and are undergoing a Quality Assurance review. The imagery data we have seen so far (NW CT) looks quite good. A nice example of the quality is the natural color imagery capture of the Kent School on the banks of the Housatonic River.

2023 aerial example, Kent School
2023 Ortho aerial imagery of The Kent School

The bare earth surface dataset, called a Digital Elevation Model (DEM), also looks great with remarkable detail. Below is an example of remnant ox bows (old river meanders) captured underneath the tree canopy that highlights geomorphology features captured by the Lidar data.

2023 DEM sample
2023 bare earth DEM

In addition, we are receiving GIS data such as the building footprints and water features. We are expecting to have around a million building footprints delivered by the end of the project that cover every town in Connecticut.

2023 GIS deliverables, Winsted example
Example of GIS deliverables, Highland Lake just west of downtown Winsted

Delivery Schedule

Flight blocks
2023 Flight Blocks and Parts

The delivery schedule has changed due to issues related to “warping” and refining the True Ortho process for cleaning up buildings in urban centers. We hope that as the Quality Assurance/Quality Control (QA/QC) process continues, the Vendor will improve their delivery speed. The new timeline is focused on completing more rural areas first followed by urban areas.

Block 1 in northwest CT was delivered first and the GIS Office will receive Block 2, part 1 and Block 4, parts 1 and 2 in the next week. The southwest CT coast will be delivered next (Block 2, part 2) along with the rest of eastern CT (Block 4, parts 3, 4, & 5). Following those, central CT (Block 3) will be delivered.

The table below includes an estimated timeline for the initial delivery to the GIS Office and final availability. For reference, following the initial delivery to the GIS Office, the following happens:

  1. the geospatial data is reviewed by the GIS Office and participating stakeholders;
  2. the review is incorporated into the dataset by the Vendor; and then
  3. the near-final dataset is returned to the GIS Office, spot checked, and is then ready for distribution.
  4. the distributors (CT ECO for aerial imagery and raster-based elevation, and the GIS Office for vector datasets) can begin work on web distribution and download which will happen as fast as possible.

In addition, 3D buildings and contours (which are derived products) will be produced after the final approval of all other data to ensure that these products have the best possible underlying quality which will be later in the summer. We appreciate your patience as we try to deliver the best possible products.

Location (type) Block Initial Vendor Delivery Estimated Availability
NW CT (imagery ) Block 1 delivered April
NW CT (GIS & elev. ) Block 1 delivered April
SW CT (imagery) Block 2, part 1 early March April-May
SW CT (imagery) Block 2, part 2 early April May
SW CT (GIS elev.) Block 2 mid-March May
Eastern CT (imagery) Block 4, parts 1,2 early March April
Eastern CT (imagery) Block 4, parts 3,4,5 April May-early June
Eastern CT (GIS & elev.) Block 4 early June July
Central CT (imagery) Block 3 May July
Central CT (GIS  & elev.) Block 3 early May July

 

CT GIS Office Newsletter Vol 1, Issue 2

GIS Office Newsletter, Volume 1 | Issue 2

GIS Office Newsletter, Volume 1, Issue 2Volume 1, Issue 2 of the GIS Office newsletter is out. It announces the new, seamless 2023 parcel layer, the 2023 Annual Report, and the new housing dashboard. It also includes updates about broadband mapping, stakeholder outreach, addressing, the strategic plan, aerial imagery acquisition update, and more.

It also provides a summary of the recent GIS Advisory Council meeting, some cool resources, and upcoming events. Check it out!

Connecticut Soil Survey Updates – Fall 2023

Soils web interfaceWritten by Jacob Isleib, State Soil Scientist

The USDA NRCS, Soil and Plant Science Division refreshes the publicly available soil survey database once a year. This Annual Soils Refresh (ASR) provides new soils data, updates to existing soils data, and new soil interpretations. The ASR also ensures all official soils data adhere to the same standards.

This year’s update to soil survey data in the State of Connecticut brings marked changes.

A summary of these changes is listed here.

1. Updated Farmland of Statewide Importance Criteria

To help maintain the productive capacity of American agriculture, the U.S. Department of Agriculture’s Natural Resources Conservation Service (NRCS) has developed criteria and guidelines to inventory important farmlands. Earlier this year, Connecticut NRCS proposed to update its farmland designation criteria for Farmland of Statewide Importance to allow for “not more than 3 percent of the soil surface is covered by stones 10 in (25 cm) to 24 in (60 cm) diameter.” This change would allow for some very stony soil map units to be designated as Farmland of Statewide Importance.  The result of these changes is a more equitable designation across Connecticut, as these designations are used to assess eligibility and ranking for certain USDA program benefits.   Further, the new designations are consistent with criteria/designations in other states of our region such as Massachusetts.

These farmland designation criteria changes result in the following changes to the Soil Survey of the State of Connecticut:

  • 339,153 acres changed to Farmland of Statewide importance. Some of this acreage was previously designated as Locally Important Farmland in select towns. This change to Farmland of Statewide importance supersedes the Locally Important Farmland designation, resulting in a more equitable designation across the state as these designations are used to assess eligibility and ranking for certain USDA program benefits.
  • 8 map units currently designated as Locally Important Farmland in select towns would not be changed to Farmland of Statewide Importance, so the Locally Important designation does not become obsolete as a result of the proposed change. These unchanged map units are associated with bedrock-controlled landforms. Connecticut and Massachusetts currently have consistent farmland designation criteria as related to amount of bedrock outcrop/exposures.

Farmland Soil class

Figure 1. The darker blue areas labeled “STATEWIDE proposed” are now considered Farmland of Statewide Importance in the FY24 soil survey data.

2. Updates to Water Table Data for Moderately Well Drained Soils

Soil water table data for some Moderately Well Drained (MWD) components in the Soil Survey for the State of Connecticut was inconsistent and, in some cases, not accurately matching soil morphology of the respective typical pedon. Additionally, the MWD components were inconsistent as to whether the month of May was included as a month with the seasonal high water table depth.  A review of available water table data suggested May should be populated as a month with a high-water table in all cases.  Many of our MWD soils were only populated through April.

Updates to the water table data will affect soil interpretation results as well as some commonly used interpretive groups such as Hydrologic Soil Groups. The following soil components have altered hydrologic soil group designations in the soil survey as a result of the water table edits: Amenia, Ashfield, Belgrade, Berlin, Brancroft, Ellington, Elmridge, Georgia, Hero, Ludlow, Ninigret, Pootatuck, Rainbow, Raypol, Schroon, Sudbury, Tisbury, Wapping, Watchaug, and Winooski.

A summary document of HSG values for previous years and the new data is available.

3. Connecticut Soil Survey Area Split into Western and Eastern Parts

Users that download the spatial and tabular soil survey data for use in GIS and/or Access will notice a significant change to how the Connecticut data is packaged. Due to new data serving limitations, the State of Connecticut (statewide) soil survey area (symbolized as CT600) needed to be split into at least two parts. The former and current soil survey areas are shown below for illustration.

Former soil boundary

Figure 3.1. Former Soil Survey Area boundary for CT600 (Soil Survey of the State of Connecticut) with MLRA boundaries

New soil boundary

Figure 3.2. New Soil Survey Area boundaries; CT601 (Soil Survey of the State of Connecticut, Western Part) and CT602 (Soil Survey of the State of Connecticut, Eastern Part)

A few criteria were used to develop the split feature (i.e., the line used to split the soil survey area into two parts). The main criteria were that the split should result in two large parts of approximately similar size and the boundary should attempt to negotiate both the legacy counties and new planning regions (aka “county equivalents”). The feature is shown in red in Figure 3.3.

Counties vs. County Equivalents

Figure 3.3. The relationship between legacy counties and county-equivalents (Planning Regions). County boundaries are shown as thick lines; names are labeled in standard font in all caps. The county-equivalents are shaded; names are in italics. Original black and white figure from US Census.

The new data can be downloaded by visiting Web Soil Survey, click “Start WSS”, select “Download Soils Data” tab, and filter by state.

4. Newly Correlated Soil Series: Pollux and Amostown Soils

Pollux and Amostown soil series were correlated within the Connecticut Valley in multiple Massachusetts soil survey areas, and are now correlated to Soil Surveys in the State of Connecticut as a result of soil survey update activities in Keney Park and Matianuck Preserve that were completed during 2023. The Pollux and Amostown series consist of very deep soils formed in loamy glacial outwash overlying glaciolacustrine sediments. Pollux is well drained and Amostown is moderately well drained.  They are nearly level to strongly sloping soils on glaciofluvial plains or deltas.

An Updated Soil Catenas of Connecticut document that includes Pollux and Amostown soils is available.

Soils web interface

Updated Soils Web Services on CT ECO and CT DEEP Open GIS

To facilitate easier access to soil survey map unit data and select popular soil interpretations, NRCS has partnered with UConn CLEAR and CT DEEP to provide web service layers of soil survey-derived data.  The newly updated layer links are available on the CT ECO Services page for:

  • All Soils – this is the soil map unit polygon data from the latest version of SSURGO
  • Farmland classification – see item 1 above for a detailed explanation of this data and how it was updated for the Fall 2023 release
  • CT Inland Wetlands
  • Hydric Soil Rating
  • Soil Parent Material Name
  • Subsurface Sewage Disposal Systems
  • Soil Drainage Class

All these new web service layers were generated using the latest version of soil survey data.

If you have any questions about new edits to Connecticut soil survey data, please contact Jacob Isleib at Jacob.isleib@usda.gov.

CT Imagery and Lidar Delivery Schedule

Written by Carl Zimmerman, PhD, GIS Coordinator, CT GIS Office (email: carl.zimmerman@ct.gov

We are all excited about the arrival of the new data and we have received questions about when it will be available. The very high resolution and quality of both the aerial imagery and Lidar create VERY LARGE files which are time-consuming to process and review. Because of this, new vendor workflows and algorithms were required and took some extra time. We know that new imagery is a top priority and the order of dataset delivery has also been altered so that imagery is received first. We are working as fast as possible while maintaining the highest data quality. 

The following describes the delivery and timeline and we will provide updates when available. 

2023 Data Delivery blocks

Delivery Blocks

The data is broken into four delivery sections: northwest CT (light blue, first), southwest CT (dark blue), the CT River Valley (green), and finally eastern CT (brown).

Schedule

Unfortunately, there is no instant gratification in the schedule. The tasks that need to be completed before the data can be finalized and provided include the review, acceptance of edits, final delivery, and then public access. Each step takes some time – there are over 20,000 imagery tiles, billions of Lidar classified points, and over a million buildings.

If you are interested in helping to review the datasets for your town or region, please reach out. There will be a web portal and we will have some training soon to demonstrate its use. 

Tentative data schedule  Block 1
Northwest CT
Block 2
Southwest CT
Block 3
Central CT
Block 4
Eastern CT
Imagery
1. Initial data delivery via online review tool 12/8/2023 1/19/2024 2/28/2024 3/31/2024
2. Data review* (QAQC**) deadline 1/10/2024 2/14/2024 3/27/2024 5/1/2024
3. Final imagery delivery to GIS Office 1/25/2024 3/1/2024 4/15/2024 5/15/2024
Lidar and GIS Data
4. Initial data delivery via online review tool 1/31/2024 2/29/2024 4/30/2024 5/31/2024
5. Data review* (QAQC**) deadline 3/1/2024 4/1/2024 6/1/2024 7/1/2024
6. Final Lidar and GIS data delivery to GIS Office 4/1/2024 5/1/2024 7/1/2024 8/1/2024

*Data review will be lead by the GIS Office with help from the Data Acquisition Working Group and stakeholders
**QAQC stands for Quality Assurance and Quality Control

2023 Data Acquisition Update

Hi CT GIS Community:

This is Carl Zimmerman (GIS Coordinator) from the CT GIS Office, and I wanted to give you a quick update on the 2023 imagery acquisition that many of you are wondering about. After some technical and quality delays around the many buy-ups of this complicated and high-resolution data capture (e.g., Imagery, LiDAR, elevation, and GIS products (such as 3d buildings)), we have a firm schedule.

Imagery will be delivered in four blocks this Spring, and the Lidar and GIS products will follow approximately two months behind the imagery in a similar pattern. Expect to see some draft imagery from CT ECO starting in early Spring and more added over the following few months. Lidar, elevation, and GIS Products will come late in the Spring and early Summer.

For us to have the best quality for this very large data set(s), four critical steps need to be finished: 1) the imagery data will be QA/QC’d by an internal subcontractor for the Vendor; 2) the GIS Office and GIS Community will review it; 3) the identified errors will be corrected by the Vendor; and 4) the finished data will be put into web services or downloadable format for you to consume. Please be patient as we get moving on this large and detailed project. More updates to follow on our progress.

Finally, I will be reaching out to some of you in December and the early Winter for your assistance in reviewing the data. If interested in helping with this large project let Alfredo Herrera or myself know.

Thanks,
Carl

Alfredo.herrera@ct.gov (CT GIO)
Carl.zimmerman@ct.gov 

CT GIS Office Newsletter

GIS Office Quarterly Newsletter, Volume 1 | Issue 1

GIS Office Newsletter, Volume 1, Issue 1The very first newsletter of the CT GIS Office is out! It includes exciting data updates about aerial imagery data acquisitions, parcel and CAMA data, and broadband mapping.

It explains the Geospatial Strategic Plan including the five strategic goals and their related activities and objectives.

It also provides a summary of the recent GIS Advisory Council meeting, some cool resources, and upcoming events. Check it out!

CT ECO System Upgrades

CT ECO has received a number of upgrades this summer and there are a couple more in the pipeline. CT ECO is a system of five servers including a web server, three mapping servers, a database server, and a large storage drive. They connect to each other so that loads of Connecticut’s geographic data can make it’s way to you on both CT ECO and the CT Geodata Portal. Along with some security patches and other minor things, the major upgrades so far are –

  1. operating systems upgraded on all of the servers which are now running the latest Windows Server 2022,
  2. software upgraded bringing the mapping servers to ArcGIS Server version 10.9.1,
  3. with the software upgrades, we republished all services under the ArcPro runtime (previously ArcMap runtime), and
  4. increased the number of processing cores in order to increase the drawing speed and reliability of services.

We are also working on upgrading the large storage drive to increase access speed and provide more room for all the 2023 files and tiles that we will be here before we know it. The team, including UConn IT attempted, this twice (gulp) with issues that we are working through before attempting again on on Tuesday, Sept 5 at 5pm.

And also coming soon will be another software upgrade to ArcGIS Server version 11.1, the latest and greatest.

All of these upgrades have resulted in periodic system downtime and interruptions in service. A number of UConn IT folks work closely with the CT ECO mapping folks to bring the worlds together and minimize impact for our users. We greatly appreciate your patience while we navigate running and maintaining a complicated system.  As always, feel free to email us at clear@uconn.edu with any issues or questions.