types
Data types used in describing Iceberg schemas.
This module implements the data types described in the Iceberg specification for Iceberg schemas. To describe an Iceberg table schema, these classes can be used in the construction of a StructType instance.
Example
str(StructType( ... NestedField(1, "required_field", StringType(), True), ... NestedField(2, "optional_field", IntegerType()) ... )) 'struct<1: required_field: required string, 2: optional_field: optional int>'
Notes
- https://iceberg.apache.org/spec/#primitive-types
BinaryType
¶
Bases: PrimitiveType
A Binary data type in Iceberg can be represented using an instance of this class.
Binaries in Iceberg are arbitrary-length byte arrays.
Example
column_foo = BinaryType() isinstance(column_foo, BinaryType) True column_foo BinaryType()
Source code in pyiceberg/types.py
BooleanType
¶
Bases: PrimitiveType
A boolean data type in Iceberg can be represented using an instance of this class.
Example
column_foo = BooleanType() isinstance(column_foo, BooleanType) True column_foo BooleanType()
Source code in pyiceberg/types.py
DateType
¶
Bases: PrimitiveType
A Date data type in Iceberg can be represented using an instance of this class.
Dates in Iceberg are calendar dates without a timezone or time.
Example
column_foo = DateType() isinstance(column_foo, DateType) True column_foo DateType()
Source code in pyiceberg/types.py
DecimalType
¶
Bases: PrimitiveType
A decimal data type in Iceberg.
Example
DecimalType(32, 3) DecimalType(precision=32, scale=3) DecimalType(8, 3) == DecimalType(8, 3) True
Source code in pyiceberg/types.py
DoubleType
¶
Bases: PrimitiveType
A Double data type in Iceberg can be represented using an instance of this class.
Doubles in Iceberg are 64-bit IEEE 754 floating points.
Example
column_foo = DoubleType() isinstance(column_foo, DoubleType) True column_foo DoubleType()
Source code in pyiceberg/types.py
FixedType
¶
Bases: PrimitiveType
A fixed data type in Iceberg.
Example
FixedType(8) FixedType(length=8) FixedType(8) == FixedType(8) True FixedType(19) == FixedType(25) False
Source code in pyiceberg/types.py
FloatType
¶
Bases: PrimitiveType
A Float data type in Iceberg can be represented using an instance of this class.
Floats in Iceberg are 32-bit IEEE 754 floating points and can be promoted to Doubles.
Example
column_foo = FloatType() isinstance(column_foo, FloatType) True column_foo FloatType()
Attributes:
| Name | Type | Description |
|---|---|---|
max |
float
|
The maximum allowed value for Floats, inherited from the canonical Iceberg implementation
in Java. (returns |
min |
float
|
The minimum allowed value for Floats, inherited from the canonical Iceberg implementation
in Java (returns |
Source code in pyiceberg/types.py
GeographyType
¶
Bases: PrimitiveType
A geography data type in Iceberg (v3+) for storing spatial geographies.
Geographies are stored as WKB (Well-Known Binary) at runtime. The CRS (Coordinate Reference System) and algorithm parameters specify the spatial reference system and the algorithm used for geographic calculations.
Example
column_foo = GeographyType() isinstance(column_foo, GeographyType) True column_foo GeographyType() column_foo.crs 'OGC:CRS84' column_foo.algorithm 'spherical' GeographyType("EPSG:4326", "planar") GeographyType(crs='EPSG:4326', algorithm='planar') str(GeographyType("EPSG:4326", "planar")) "geography('EPSG:4326', 'planar')"
Source code in pyiceberg/types.py
1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 | |
algorithm
property
¶
Return the algorithm used for geographic calculations.
crs
property
¶
Return the CRS (Coordinate Reference System) of the geography.
__eq__(other)
¶
__getnewargs__()
¶
__hash__()
¶
__repr__()
¶
Return the string representation of the GeographyType class.
Source code in pyiceberg/types.py
__str__()
¶
Return the string representation.
Source code in pyiceberg/types.py
minimum_format_version()
¶
ser_model()
¶
Serialize the model to a string.
Source code in pyiceberg/types.py
GeometryType
¶
Bases: PrimitiveType
A geometry data type in Iceberg (v3+) for storing spatial geometries.
Geometries are stored as WKB (Well-Known Binary) at runtime. The CRS (Coordinate Reference System) parameter specifies the spatial reference system for the geometry values.
Example
column_foo = GeometryType() isinstance(column_foo, GeometryType) True column_foo GeometryType() column_foo.crs 'OGC:CRS84' GeometryType("EPSG:4326") GeometryType(crs='EPSG:4326') str(GeometryType("EPSG:4326")) "geometry('EPSG:4326')"
Source code in pyiceberg/types.py
IcebergType
¶
Bases: IcebergBaseModel
Base type for all Iceberg Types.
Example
str(IcebergType()) 'IcebergType()' repr(IcebergType()) 'IcebergType()'
Source code in pyiceberg/types.py
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 | |
IntegerType
¶
Bases: PrimitiveType
An Integer data type in Iceberg can be represented using an instance of this class.
Integers in Iceberg are 32-bit signed and can be promoted to Longs.
Example
column_foo = IntegerType() isinstance(column_foo, IntegerType) True
Attributes:
| Name | Type | Description |
|---|---|---|
max |
int
|
The maximum allowed value for Integers, inherited from the canonical Iceberg implementation
in Java (returns |
min |
int
|
The minimum allowed value for Integers, inherited from the canonical Iceberg implementation
in Java (returns |
Source code in pyiceberg/types.py
ListType
¶
Bases: IcebergType
A list type in Iceberg.
Example
ListType(element_id=3, element_type=StringType(), element_required=True) ListType(element_id=3, element_type=StringType(), element_required=True)
Source code in pyiceberg/types.py
LongType
¶
Bases: PrimitiveType
A Long data type in Iceberg can be represented using an instance of this class.
Longs in Iceberg are 64-bit signed integers.
Example
column_foo = LongType() isinstance(column_foo, LongType) True column_foo LongType() str(column_foo) 'long'
Attributes:
| Name | Type | Description |
|---|---|---|
max |
int
|
The maximum allowed value for Longs, inherited from the canonical Iceberg implementation
in Java. (returns |
min |
int
|
The minimum allowed value for Longs, inherited from the canonical Iceberg implementation
in Java (returns |
Source code in pyiceberg/types.py
MapType
¶
Bases: IcebergType
A map type in Iceberg.
Example
MapType(key_id=1, key_type=StringType(), value_id=2, value_type=IntegerType(), value_required=True) MapType(key_id=1, key_type=StringType(), value_id=2, value_type=IntegerType(), value_required=True)
Source code in pyiceberg/types.py
NestedField
¶
Bases: IcebergType
Represents a field of a struct, a map key, a map value, or a list element.
This is where field IDs, names, docs, and nullability are tracked.
Example
str(NestedField( ... field_id=1, ... name='foo', ... field_type=FixedType(22), ... required=False, ... )) '1: foo: optional fixed[22]' str(NestedField( ... field_id=2, ... name='bar', ... field_type=LongType(), ... is_optional=False, ... doc="Just a long" ... )) '2: bar: required long (Just a long)' str(NestedField( ... field_id=3, ... name='baz', ... field_type="string", ... required=True, ... doc="A string field" ... )) '3: baz: required string (A string field)'
Source code in pyiceberg/types.py
385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 | |
__getnewargs__()
¶
__repr__()
¶
Return the string representation of the NestedField class.
Source code in pyiceberg/types.py
__str__()
¶
Return the string representation of the NestedField class.
Source code in pyiceberg/types.py
convert_field_type(v)
¶
Convert string values into IcebergType instances.
Source code in pyiceberg/types.py
PrimitiveType
¶
Bases: Singleton, IcebergRootModel[str], IcebergType
Base class for all Iceberg Primitive Types.
Source code in pyiceberg/types.py
__repr__()
¶
StringType
¶
Bases: PrimitiveType
A String data type in Iceberg can be represented using an instance of this class.
Strings in Iceberg are arbitrary-length character sequences and are encoded with UTF-8.
Example
column_foo = StringType() isinstance(column_foo, StringType) True column_foo StringType()
Source code in pyiceberg/types.py
StructType
¶
Bases: IcebergType
A struct type in Iceberg.
Example
str(StructType( ... NestedField(1, "required_field", StringType(), True), ... NestedField(2, "optional_field", IntegerType()) ... )) 'struct<1: required_field: optional string, 2: optional_field: optional int>'
Source code in pyiceberg/types.py
TimeType
¶
Bases: PrimitiveType
A Time data type in Iceberg can be represented using an instance of this class.
Times in Iceberg have microsecond precision and are a time of day without a date or timezone.
Example
column_foo = TimeType() isinstance(column_foo, TimeType) True column_foo TimeType()
Source code in pyiceberg/types.py
TimestampNanoType
¶
Bases: PrimitiveType
A TimestampNano data type in Iceberg can be represented using an instance of this class.
TimestampNanos in Iceberg have nanosecond precision and include a date and a time of day without a timezone.
Example
column_foo = TimestampNanoType() isinstance(column_foo, TimestampNanoType) True column_foo TimestampNanoType()
Source code in pyiceberg/types.py
TimestampType
¶
Bases: PrimitiveType
A Timestamp data type in Iceberg can be represented using an instance of this class.
Timestamps in Iceberg have microsecond precision and include a date and a time of day without a timezone.
Example
column_foo = TimestampType() isinstance(column_foo, TimestampType) True column_foo TimestampType()
Source code in pyiceberg/types.py
TimestamptzNanoType
¶
Bases: PrimitiveType
A TimestamptzNano data type in Iceberg can be represented using an instance of this class.
TimestamptzNanos in Iceberg are stored as UTC and include a date and a time of day with a timezone.
Example
column_foo = TimestamptzNanoType() isinstance(column_foo, TimestamptzNanoType) True column_foo TimestamptzNanoType()
Source code in pyiceberg/types.py
TimestamptzType
¶
Bases: PrimitiveType
A Timestamptz data type in Iceberg can be represented using an instance of this class.
Timestamptzs in Iceberg are stored as UTC and include a date and a time of day with a timezone.
Example
column_foo = TimestamptzType() isinstance(column_foo, TimestamptzType) True column_foo TimestamptzType()
Source code in pyiceberg/types.py
UUIDType
¶
Bases: PrimitiveType
A UUID data type in Iceberg can be represented using an instance of this class.
UUIDs in Iceberg are universally unique identifiers.
Example
column_foo = UUIDType() isinstance(column_foo, UUIDType) True column_foo UUIDType()
Source code in pyiceberg/types.py
UnknownType
¶
Bases: PrimitiveType
An unknown data type in Iceberg can be represented using an instance of this class.
Unknowns in Iceberg are used to represent data types that are not known at the time of writing.
Example
column_foo = UnknownType() isinstance(column_foo, UnknownType) True column_foo UnknownType()
Source code in pyiceberg/types.py
strtobool(val)
¶
Convert a string representation of truth to true (1) or false (0).
True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if 'val' is anything else.
Source code in pyiceberg/types.py
transform_dict_value_to_str(dict)
¶
Transform all values in the dictionary to string. Raise an error if any value is None.