Lithology Cleaning and Categorizing

Overview

When GWELLS data is downloaded, the raw lithology descriptions are cleaned and categories into lithology categories for use by other programs.

This results in ultimately transforming the original starting description from GWELLS into a new lithology category defined by a set of rules.

For example, an original lithology record of “gravel with some sandy seams” would be categorized as “Sand and Gravel (Clean)”.

However this process happens over several steps and for transparency, the outputs of intermediate steps are retained in the final data set.

Here is a full example of lithology data.

lithology_raw_combined	lithology_clean	lith_primary	lith_secondary	lith_tertiary	lithology_extra	lithology_category	flag_cat_bedrock	flag_cat_boulders	flag_cat_missing
gravl w/ sands	gravel with sand	gravel	sand			Sand and Gravel (Clean)	FALSE	FALSE	FALSE
bentonite						NA	FALSE	FALSE	TRUE
sand and roots	sand & organic	sand, organic			organic	Organics	FALSE	FALSE	FALSE
muddy sand	silty sand	sand		silt		Sand and Fines	FALSE	FALSE	FALSE
reddish sand with pink gravel	sand with gravel	sand	gravel			Sand and Gravel (Clean)	FALSE	FALSE	FALSE

In this article we will explain how this data is created.

Categorization Steps

Categorizing lithology happens over three steps:

1. Cleaning
2. Initial categorizing
3. Final categorizing

The lithology data contains columns reflecting these steps.

Column	Description	Step
lithology_raw_combined	Original lithology descriptions from GWELLS (combination of lithology_raw_data, lithology_description_code and lithology_material_code)	Original Data
lithology_clean	Cleaned lithology description	1. Cleaning
lith_primary, lith_secondary, lith_tertiary	Intermediate categories created from lithology_clean	2. Initial categorizing
lithology_extra	Extra, potentially important descriptors extracted from the lithology description	3. Final categorizing
lithology_category	Final categorized lithology	3. Final categorizing
flag_cat_bedrock, flag_cat_boulders, flag_cat_missing	Columns flagging a particular observation as problematic	3. Final categorizing

1. Cleaning

• Remove erroneous text (unnecessary qualifiers)
• Fix spelling mistakes
• Consolidate/standardize similar terms

For example…

Starting Description (lithology_raw_combined)	Cleaned Description (lithology_clean)
gravl w/ sands	gravel with sand
bentonite
sand and roots	sand & organic
muddy sand	silty sand
reddish sand with pink gravel	sand with gravel

2. Initial Categorizing

Create primary, secondary and tertiary categories from important terms

Primary categories

• ‘Standalone’ terms, possibly qualified by other categories
• e.g., sand, silt, clay, till, boulders, bedrock

Secondary categories

• ‘With’ terms
• e.g., with sand, with silt, with clay, with boulders, with bedrock, with till

Tertiary categories

• Terms ending in ‘y’/‘ey’
• sandy, silty, clayey, tilly, bouldery

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)
	1°	2°	3°
sand	sand
sand with silt	sand	silt
silty sand	sand		silt
sand with silty clay	sand, clay		silt
Note: There can be multiple terms per category.

3. Final Categorizing

These categories are then used to define a single final category, according to a a set of rules

For example…

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
sand	sand			Sand
sand with silt	sand	silt		Sand and Fines
silty sand	sand		silt	Sand and Fines
sand with silty clay	sand, clay		silt	Sand or Gravel Till or Diamicton

The “Categorization” section explains in more detail how this final category is decided upon.

Flags and Extra

In addition to creating the lithology category, we flag specific situations that may warrent extra investigation, as well as pull out and note some terms in an ‘extra’ column (lithology_extra).

Categorization Rules

Here are the rules used to define final lithology categories, by examining the primary, secondary, and tertiary categories.

Note: These rules are in order of importance. Therefore if a combination of terms matches more than one rule, the first rule takes presidence.

Weathered, Fractured or Faulted Bedrock

Any category is fractured, weathered, or faulted

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)	Flags
	1°	2°	3°		bedrock	boulders
fractured bedrock	fractured, bedrock			Weathered, Fractured or Faulted Bedrock
fractured	fractured			Weathered, Fractured or Faulted Bedrock
fractured bedrock and sand	fractured, bedrock, sand			Weathered, Fractured or Faulted Bedrock	TRUE
Flags: Flagged if present with any other primary categories except bedrock

Bedrock

Any category is bedrock

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)	Flags
	1°	2°	3°		bedrock	boulders
bedrock	bedrock			Bedrock
sand with bedrock	sand	bedrock		Bedrock	TRUE
sandy bedrock	bedrock		sand	Bedrock
sand & bedrock	sand, bedrock			Bedrock	TRUE
Flags: Flagged if present with any other primary categories

Boulders

Any category is boulders

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)	Flags
	1°	2°	3°		bedrock	boulders
boulders	boulders			Boulders
sand with boulders	sand	boulders		Boulders		TRUE
sandy boulders	boulders		sand	Boulders
sand & boulders	sand, boulders			Boulders		TRUE
bouldery			boulders	Boulders
bouldery sand	sand		boulders	Boulders		TRUE
hardpan, gravel, boulders1	hardpan, gravel, boulders			Medium to Clay Till or Diamicton		TRUE
Flags: Flagged if present with any other primary categories Extra: Noted in ‘Extra’ column (lithology_extra) if present in any category
1 Example of similar terms but different category

Organics

Primary is organic

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
organic	organic			Organics
organic with sand	organic	sand		Organics
sandy organic	organic		sand	Organics
organic and sand	organic, sand			Organics
Extra: Noted in ‘Extra’ column (lithology_extra) if present in any category

Gravel, Sand, Clay, or Silt

Primary is gravel, sand, clay, or silt
No Secondary/Tertiary (Except silty clay and clay with silt)

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
gravel	gravel			Gravel
sand	sand			Sand
clay	clay			Clay
silt	silt			Silt
silty clay	clay		silt	Clay
clay with silt	clay	silt		Clay

Sandy or Gravelly Silt

Primary is silt
Secondary/Tertiary are sand or gravel

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
silt with sand	silt	sand		Sandy or Gravelly Silt
silt with gravel	silt	gravel		Sandy or Gravelly Silt
sandy silt	silt		sand	Sandy or Gravelly Silt
gravely silt	silt		gravel	Sandy or Gravelly Silt
gravely silt with sand	silt	sand	gravel	Sandy or Gravelly Silt

Sand and Gravel (Clean)

Both (and only) sand and gravel are both present in any category

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
sand & gravel	sand, gravel			Sand and Gravel (Clean)
gravel & sand	gravel, sand			Sand and Gravel (Clean)
sand, gravel	sand, gravel			Sand and Gravel (Clean)
sandy with gravel		gravel	sand	Sand and Gravel (Clean)
sand & gravel & silt1	sand, gravel, silt			Sand and Gravel (Dirty)
1 Example of similar terms but different category

Sand and Gravel (Dirty)

gravel or sand are both present in any category, at least one is Primary and Secondary/Tertiary is also silt or clay
OR
Primary is all gravel, sand and silt/clay

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
gravel & sand with silt	gravel, sand	silt		Sand and Gravel (Dirty)
gravel & sand with clay	gravel, sand	clay		Sand and Gravel (Dirty)
silty gravel & sand	gravel, sand		silt	Sand and Gravel (Dirty)
clayey gravel & sand	gravel, sand		clay	Sand and Gravel (Dirty)
gravel & sand & silt	gravel, sand, silt			Sand and Gravel (Dirty)
gravel & sand & clay	gravel, sand, clay			Sand and Gravel (Dirty)
gravelly sand with silt	sand	silt	gravel	Sand and Gravel (Dirty)
clayey sand with gravel	sand	gravel	clay	Sand and Gravel (Dirty)

Sand and Fines

Primary is sand and Secondary/Tertiary is silt or clay
OR
Primary is both sand and silt (not clay)

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
sand with silt	sand	silt		Sand and Fines
sand with clay	sand	clay		Sand and Fines
silty sand	sand		silt	Sand and Fines
clayey sand	sand		clay	Sand and Fines
sand & silt	sand, silt			Sand and Fines
gravelly sand with silt1	sand	silt	gravel	Sand and Gravel (Dirty)
sand & clay1	sand, clay			Sand or Gravel Till or Diamicton
1 Example of similar terms but different category

Gravel (Dirty)

Primary is gravel and Secondary/Tertiary is silt or clay
OR
Primary is both gravel and silt (not clay)

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
gravel with silt	gravel	silt		Gravel (Dirty)
gravel with clay	gravel	clay		Gravel (Dirty)
silty gravel	gravel		silt	Gravel (Dirty)
clayey gravel	gravel		clay	Gravel (Dirty)
gravel & silt	gravel, silt			Gravel (Dirty)
gravel & clay1	gravel, clay			Sand or Gravel Till or Diamicton
1 Example of similar terms but different category

Sand or Gravel Till or Diamicton

Any category is sgtill
OR
Primary is till or clay and any category is sand or gravel (but both cannot be primary) OR
Primary is sand or gravel and Secondary/Tertiary is till
OR
Primary is compact and any category is sand or gravel

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
sgtill	sgtill			Sand or Gravel Till or Diamicton
till with gravel and sand seam	till	gravel	sand	Sand or Gravel Till or Diamicton
till with sand	till	sand		Sand or Gravel Till or Diamicton
clay with gravel	clay	gravel		Sand or Gravel Till or Diamicton
sand with till	sand	till		Sand or Gravel Till or Diamicton
tilly gravel	gravel		till	Sand or Gravel Till or Diamicton
compact sand	compact, sand			Sand or Gravel Till or Diamicton
compact gravel	compact, gravel			Sand or Gravel Till or Diamicton
compact with sand	compact	sand		Sand or Gravel Till or Diamicton
clay & gravel	clay, gravel			Sand or Gravel Till or Diamicton
clay & sand	clay, sand			Sand or Gravel Till or Diamicton
gravel & sand & silt1	gravel, sand, silt			Sand and Gravel (Dirty)
1 Example of similar terms but different category

Medium to Clay Till or Diamicton

Primary is till, hardpan or hard earth
OR
Primary is silt and Secondary/Tertiary is till
OR
Primary is clay and any category is till
OR
Primary is compact and any category is silt or clay
OR
Any combination of silt or clay not already categorized

Note: That silty clay is already categorized as “Clay” (see Gravel, Sand, Clay, or Silt)

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
till	till			Medium to Clay Till or Diamicton
hard pan	hardpan			Medium to Clay Till or Diamicton
hard earth	hard earth			Medium to Clay Till or Diamicton
silt with till	silt	till		Medium to Clay Till or Diamicton
tilly silt	silt		till	Medium to Clay Till or Diamicton
clay & till	clay, till			Medium to Clay Till or Diamicton
tilly clay	clay		till	Medium to Clay Till or Diamicton
compact silt	compact, silt			Medium to Clay Till or Diamicton
compact clay	compact, clay			Medium to Clay Till or Diamicton
compact with silt	compact	silt		Medium to Clay Till or Diamicton
compact with clay	compact	clay		Medium to Clay Till or Diamicton
silt & clay	silt, clay			Medium to Clay Till or Diamicton
silt with clay	silt	clay		Medium to Clay Till or Diamicton
clayey silt	silt		clay	Medium to Clay Till or Diamicton
silty clay1	clay		silt	Clay
clay with silt1	clay	silt		Clay
1 Example of similar terms but different category

Shells

Primary is only shells
No Secondary/Tertiary

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
shells	shells			Shells
shells & sand1	shells, sand			Sand
Extra: Noted in ‘Extra’ column (lithology_extra) if present in any category
1 Example of similar terms but different category

Overburden

Primary is only overburden
No Secondary/Tertiary

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
overburden	overburden			Overburden
overburden & sand1	overburden, sand			Sand
1 Example of similar terms but different category

No category

All categories are empty

Extra columns

• Organics, Boulders, and Shells are noted in the column lithology_extra

• As are:

  • flow (water, flowing, stream of water, etc.)
  • seepage, wet, saturated, trickle
  • waterbearing (water-bearing, wb, w.b. etc.)
  • aquifer, reservoir, artesian

Ambiguous distinctions

Rock vs. rocks. vs rocky

Starting Description (lithology_raw_combined)	Intermediate Categories (lith_primary,lith_secondary,lith_tertiary)			Final Category (lithology_category)
	1°	2°	3°
broken rock	fractured, bedrock			Weathered, Fractured or Faulted Bedrock
rock	bedrock			Bedrock
rocks	gravel			Gravel
rocky			gravel	Gravel