Skip to main content
ecuscino | Created: 12 Dec 2024 | Updated: 12 Dec 2024
Categories
Blog
DIGGS News
Tags
DIGGS Blog
DIGGS

Note: This work was completed as part of the Vendor Support portion of the ASCE Geo-Institute Technical Committee Special Project Fund.

The Challenge of Piezometer Data

Piezometers typically produce data in a simple, time-series format, usually stored in CSV files. A typical example looks like this:

Sensor,Date,Pressure (ft H₂O),Elevation H₂O (ft),Temperature (deg F),Latitude,Longitude,Depth
PZ-B-03-23,2024-11-5 13:00:01,23.683084983767312,774.1830849837672,56.97303791760282,38.01979088794593,-83.84532852561108,8.382

This format is straightforward: each row represents a measurement from a specific sensor at a given time, including key parameters like pressure, water elevation, temperature, and location. This is a very common format for any continuously collected geotechnical dataset. While manageable for individual datasets, this approach can become unwieldy when dealing with multiple piezometers or long-term monitoring projects. You might find yourself with numerous CSV files, each needing to be processed separately.

Introducing DIGGS for Standardized Data Management

DIGGS offers a more robust solution. It's an industry-standard XML-based format that provides a structured way to store geotechnical data, making it easier to share, analyze, and archive.

To simplify the transition to DIGGS, we've developed a user-friendly converter, available on GitHub: DIGGS Piezometer Converter. This tool takes your CSV data and transforms it into DIGGS format, ready for use in various geotechnical software and analysis platforms.

Understanding How DIGGS Stores Continuous Data

Let's delve into how DIGGS handles continuous data like piezometer readings. Here's a snippet from DIGGS file representing piezometer data readings:

<reading>
        <Reading gml:id="Reading_Water_Levels_PZ-B-03-23">
            <responseZoneLocation xlink:href="#rzl"/>
            <outcome>
                <MonitorResult gml:id="Result_Water_Levels_PZ-B-03-23">
                    <timeDomain>
                        <TimePositionList gml:id="TP_Water_Levels_PZ-B-03-23">
                            <timePositionList>2024-11-05T13:00:01 2024-11-05T12:30:01</timePositionList>
                        </TimePositionList>
                    </timeDomain>
                    <results>
                        <ResultSet>
                            <parameters>
                                <PropertyParameters gml:id="pp_Water_Levels_PZ-B-03-23_1">
                                    <properties>
                                        <Property index="1" gml:id="prop1_Water_Levels_PZ-B-03-23">
                                            <propertyName>Pressure (ft H₂O)</propertyName>
                                            <typeData>double</typeData>
                                            <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">water_depth</propertyClass>
                                            <uom>ft</uom>
                                        </Property>
                                        <Property index="2" gml:id="prop2_Water_Levels_PZ-B-03-23">
                                            <propertyName>Elevation H₂O (ft)</propertyName>
                                            <typeData>double</typeData>
                                            <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">water_elev</propertyClass>
                                            <uom>ft</uom>
                                        </Property>
                                        <Property index="3" gml:id="prop3_Water_Levels_PZ-B-03-23">
                                            <propertyName>Temperature (deg F)</propertyName>
                                            <typeData>double</typeData>
                                            <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">temperature</propertyClass>
                                            <uom>degF</uom>
                                        </Property>
                                    </properties>
                                </PropertyParameters>
                            </parameters>
                            <dataValues>23.683084983767312,774.1830849837672,56.97303791760282 23.644656287609216,774.1446562876092,57.01296621093711</dataValues>
                        </ResultSet>
                    </results>
                </MonitorResult>
            </outcome>
            <sensor>
                <Sensor gml:id="PZ-B-03-23">
                    <gml:name>PZ-B-03-23</gml:name>
                    <class>Pressure transducer</class>
                    <detector>
                        <Detector gml:id="metric">
                            <measurand codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">water_depth_calc</measurand>
                        </Detector>
                    </detector>
                </Sensor>
            </sensor>
        </Reading>
    </reading>

 

View complete DIGGS XML file: modified_PZ-B-03-23.xml

 

Breaking Down the DIGGS Structure

While the XML structure may look complex at first glance, it follows a logical organization that we can break down into key components:

  1. Time: The <TimePositionList> element stores the timestamps associated with your measurements. Each time point is clearly defined, ensuring accurate temporal analysis. DIGGS requires ISO 8601 time format and uses a space " " between readings.

    <TimePositionList gml:id="TP_Water_Levels_PZ-B-03-23">
        <timePositionList>2024-11-05T13:00:01 2024-11-05T12:30:01</timePositionList>
    </TimePositionList>
  2. Properties: The <PropertyParameters> section defines the types of measurements collected. Each property includes its name, data type, standardized code, and unit of measure:

    <properties>
                    <Property index="1" gml:id="prop1_Water_Levels_PZ-B-03-23">
                        <propertyName>Pressure (ft H₂O)</propertyName>
                        <typeData>double</typeData>
                        <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">water_depth</propertyClass>
                        <uom>ft</uom>
                    </Property>
                    <Property index="2" gml:id="prop2_Water_Levels_PZ-B-03-23">
                        <propertyName>Elevation H₂O (ft)</propertyName>
                        <typeData>double</typeData>
                        <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">water_elev</propertyClass>
                        <uom>ft</uom>
                    </Property>
                    <Property index="3" gml:id="prop3_Water_Levels_PZ-B-03-23">
                        <propertyName>Temperature (deg F)</propertyName>
                        <typeData>double</typeData>
                        <propertyClass codeSpace="https://diggsml.org/def/codes/DIGGS/0.1/properties.xml">temperature</propertyClass>
                        <uom>degF</uom>
                    </Property>
                </properties>
  3. Data Values: The <dataValues> element contains the actual measurements.

    <dataValues>23.683084983767312,774.1830849837672,56.97303791760282 23.644656287609216,774.1446562876092,57.01296621093711</dataValues>

From DIGGS to Visualization: Bringing Your Data to Life

The real power of DIGGS comes from its ability to facilitate data analysis and visualization. We've included a Jupyter Notebook called visualize_data.ipynb in the GitHub repository that demonstrates how to create interactive GIS maps from your DIGGS piezometer data.

The code reads all your DIGGS files, extracts the location (latitude and longitude), time, and water elevation data. Then, it generates a dynamic map using Plotly, where you can visualize the water elevation at each piezometer location over time. You can scrub through the timeline and see how water levels change across your site. The code is designed to work with multiple DIGGS files, so you can easily visualize data from all your piezometers in a single, interactive map.

Interactive visualization: Use the timeline controls to see how water levels change over time. You can zoom, pan, and hover over points for detailed information.

Conclusion

DIGGS provides a powerful and standardized way to manage geotechnical data, including piezometer readings. By converting your CSV data to DIGGS using our converter and leveraging the provided visualization tools, you can streamline your data management workflow, improve data sharing, and gain valuable insights from interactive visualizations.

We encourage you to explore the GitHub repository, try out the converter and visualization tools, and start incorporating DIGGS into your geotechnical projects! Please feel free to contribute to the repository with issues or pull requests to help expand this powerful open source tool.


Ross Cutts, P.E., M.ASCE, Part 107 Pilot

President, Geosetta

Senior Engineer
Research, Development, and Innovation (RDI) Group
Schnabel Engineering