Using Data Adapters¶
This is an index of all the available data adapters, both savers and loaders. Note that some savers and loaders are the same (certain classes can handle both), but some are different. You will want to reference this when calling out to any of the following:
Using load_from [or for just exposing metadata dataloader].
Using materializers.
To read these tables, you want to first look at the key to determine which format you want – these should be human-readable and familiar to you. Then you’ll want to look at the types field to figure out which is the best for your case (the object you want to load from or save to).
Finally, look up the adapter params to see what parameters you can pass to the data adapters. The optional params come with their default value specified.
If you want more information, click on the module, it will send you to the code that implements it to see how the parameters are used.
As an example, say we wanted to save a pandas dataframe to a CSV file. We would first find the key csv, which would inform us that we want to call save_to.csv (or to.csv in the case of materialize). Then, we would look at the types field, finding that there is a pandas dataframe adapter. Finally, we would look at the params field, finding that we can pass path, and (optionally) sep (which we’d realize defaults to , when looking at the code).
All together, we’d end up with:
import pandas as pd
from hamilton.function_modifiers import value, save_to
@save_to.csv(path=value("my_file.csv"))
def my_data(...) -> pd.DataFrame:
...
For a less “abstracted” approach, where you just expose metadata from saving and loading, you can annotated your saving/loading functions to do so, e.g. analogous to the above you could do:
import pandas as pd
from hamilton.function_modifiers import datasaver
def my_data(...) -> pd.DataFrame:
# your function
...
return _df # return some df
@datasaver
def my_data_saver(my_data: pd.DataFrame, path: str) -> dict:
# code to save my_data
return {"path": path, "type": "csv", ...} # add other metadata
See dataloader for more information on how to load data and expose metadata via this more lighter weight way.
If you want to extend the @save_to or @load_from decorators, see Using Data Adapters for documentation, and the example in the repository for an example of how to do so.
Note that you will need to call registry.register_adapters (or import a module that does that) prior to dynamically referring to these in the code – otherwise we won’t know about them, and won’t be able to access that key!
Data Loaders¶
key |
loader params |
types |
module |
|---|---|---|---|
| json |
str |
dictlist |
hamilton.io.default_data_loaders |
| json |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderchunksize int | None=Nonecompression str | dict[str, typing.Any] | None=inferconvert_axes bool | None=Noneconvert_dates bool | list[str]=Truedate_unit str | None=Nonedtype Union=Nonedtype_backend str | None=Noneencoding str | None=Noneencoding_errors str | None=strictengine str=ujsonkeep_default_dates bool=Truelines bool=Falsenrows int | None=Noneorient str | None=Noneprecise_float bool=Falsestorage_options dict[str, typing.Any] | None=Nonetyp str=frame |
DataFrame |
hamilton.plugins.pandas_extensions |
| json |
str | pathlib.Path | io.IOBase | bytesschema collections.abc.Mapping[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]] | collections.abc.Sequence[str | tuple[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]]]=Noneschema_overrides collections.abc.Mapping[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]] | collections.abc.Sequence[str | tuple[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]]]=None |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| json |
str | bytearray | os.PathLike |
XGBModelBooster |
hamilton.plugins.xgboost_extensions |
| literal |
Any |
Any |
hamilton.io.default_data_loaders |
| file |
strencoding str=utf-8 |
str |
hamilton.io.default_data_loaders |
| file |
str | pathlib.Path |
LGBMModelBoosterCVBooster |
hamilton.plugins.lightgbm_extensions |
| pickle |
str |
objectAny |
hamilton.io.default_data_loaders |
| pickle |
str | pathlib.Path | _io.BytesIO | _io.BufferedReader=Nonepath str | pathlib.Path | _io.BytesIO | _io.BufferedReader=Nonecompression str | dict[str, typing.Any] | None=inferstorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| environment |
tuple |
dict |
hamilton.io.default_data_loaders |
| yaml |
str | pathlib.Path |
strintfloatbooldictlist |
hamilton.plugins.yaml_extensions |
| npy |
str | pathlib.Path | typing.IOmmap_mode str | None=Noneallow_pickle bool | None=Nonefix_imports bool | None=Noneencoding Literal=ASCII |
ndarray |
hamilton.plugins.numpy_extensions |
| csv |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadersep str | None=,delimiter str | None=Noneheader Union=infernames collections.abc.Sequence | None=Noneindex_col Union=Noneusecols list[collections.abc.Hashable] | collections.abc.Callable | tuple | None=Nonedtype Union=Noneengine Optional=Noneconverters collections.abc.Mapping | None=Nonetrue_values list | None=Nonefalse_values list | None=Noneskipinitialspace bool | None=Falseskiprows list[int] | int | collections.abc.Callable[[collections.abc.Hashable], bool] | None=Noneskipfooter int=0nrows int | None=Nonena_values collections.abc.Hashable | collections.abc.Iterable | collections.abc.Mapping | None=Nonekeep_default_na bool=Truena_filter bool=Trueverbose bool=Falseskip_blank_lines bool=Trueparse_dates bool | collections.abc.Sequence | None=Falsekeep_date_col bool=Falsedate_format str | None=Nonedayfirst bool=Falsecache_dates bool=Trueiterator bool=Falsechunksize int | None=Nonecompression Union=inferthousands str | None=Nonedecimal str=.lineterminator str | None=Nonequotechar str | None=Nonequoting int=0doublequote bool=Trueescapechar str | None=Nonecomment str | None=Noneencoding str=utf-8encoding_errors Union=strictdialect str | csv.Dialect | None=Noneon_bad_lines Union=errordelim_whitespace bool=Falselow_memory bool=Truememory_map bool=Falsefloat_precision Optional=Nonestorage_options dict[str, typing.Any] | None=Nonedtype_backend Literal=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| csv |
str | typing.TextIO | _io.BytesIO | pathlib.Path | typing.BinaryIO | byteshas_header bool=Trueinclude_header bool=Truecolumns collections.abc.Sequence[int] | collections.abc.Sequence[str]=Nonenew_columns Sequence=Noneseparator str=,comment_char str=Nonequote_char str="skip_rows int=0dtypes collections.abc.Mapping[str, typing.Any] | collections.abc.Sequence[typing.Any]=Nonenull_values str | collections.abc.Sequence[str] | dict[str, str]=Nonemissing_utf8_is_empty_string bool=Falseignore_errors bool=Falsetry_parse_dates bool=Falsen_threads int=Noneinfer_schema_length int=100batch_size int=8192n_rows int=Noneencoding Union=utf8low_memory bool=Falserechunk bool=Trueuse_pyarrow bool=Falsestorage_options dict=Noneskip_rows_after_header int=0row_count_name str=Nonerow_count_offset int=0sample_size int=1024eol_char str=
raise_if_empty bool=True |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| csv |
str | typing.TextIO | _io.BytesIO | pathlib.Path | typing.BinaryIO | byteshas_header bool=Truecolumns collections.abc.Sequence[int] | collections.abc.Sequence[str]=Nonenew_columns Sequence=Noneseparator str=,comment_char str=Nonequote_char str="skip_rows int=0dtypes collections.abc.Mapping[str, typing.Any] | collections.abc.Sequence[typing.Any]=Nonenull_values str | collections.abc.Sequence[str] | dict[str, str]=Nonemissing_utf8_is_empty_string bool=Falseignore_errors bool=Falsetry_parse_dates bool=Falsen_threads int=Noneinfer_schema_length int=100batch_size int=8192n_rows int=Noneencoding type | str=utf8low_memory bool=Falserechunk bool=Trueuse_pyarrow bool=Falsestorage_options dict=Noneskip_rows_after_header int=0row_count_name str=Nonerow_count_offset int=0eol_char str=
raise_if_empty bool=True |
LazyFrame |
hamilton.plugins.polars_lazyframe_extensions |
| csv |
SparkSessionpath strheader bool=Truesep str=, |
DataFrame |
hamilton.plugins.spark_extensions |
| parquet |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderengine Literal=autocolumns list[str] | None=Nonestorage_options dict[str, typing.Any] | None=Noneuse_nullable_dtypes bool=Falsedtype_backend Literal=numpy_nullablefilesystem str | None=Nonefilters list[tuple] | list[list[tuple]] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| parquet |
str | typing.TextIO | _io.BytesIO | pathlib.Path | typing.BinaryIO | bytescolumns list[int] | list[str]=Nonen_rows int=Noneuse_pyarrow bool=Falsememory_map bool=Truestorage_options dict=Noneparallel Any=autorow_count_name str=Nonerow_count_offset int=0low_memory bool=Falsepyarrow_options dict=Noneuse_statistics bool=Truerechunk bool=True |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| parquet |
str | typing.TextIO | _io.BytesIO | pathlib.Path | typing.BinaryIO | bytescolumns list[int] | list[str]=Nonen_rows int=Noneuse_pyarrow bool=Falsememory_map bool=Truestorage_options dict=Noneparallel Any=autorow_count_name str=Nonerow_count_offset int=0low_memory bool=Falseuse_statistics bool=Truerechunk bool=True |
LazyFrame |
hamilton.plugins.polars_lazyframe_extensions |
| parquet |
SparkSessionpath str |
DataFrame |
hamilton.plugins.spark_extensions |
| sql |
strdb_connection str | sqlite3.Connectionchunksize int | None=Nonecoerce_float bool=Truecolumns list[str] | None=Nonedtype Union=Nonedtype_backend str | None=Noneindex_col str | list[str] | None=Noneparams list | tuple | dict | None=Noneparse_dates list | dict | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| xml |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderxpath str | None=./*namespace dict[str, str] | None=Noneelems_only bool | None=Falseattrs_only bool | None=Falsenames list[str] | None=Nonedtype dict[str, typing.Any] | None=Noneconverters dict[int | str, typing.Any] | None=Noneparse_dates bool | list[int | str | list[list] | dict[str, list[int]]]=Falseencoding str | None=utf-8parser str=lxmlstylesheet str | pathlib.Path | _io.BytesIO | _io.BufferedReader=Noneiterparse dict[str, list[str]] | None=Nonecompression str | dict[str, typing.Any] | None=inferstorage_options dict[str, typing.Any] | None=Nonedtype_backend str=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| html |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadermatch str | None=.+flavor str | collections.abc.Sequence | None=Noneheader int | collections.abc.Sequence | None=Noneindex_col int | collections.abc.Sequence | None=Noneskiprows int | collections.abc.Sequence | slice | None=Noneattrs dict[str, str] | None=Noneparse_dates bool | None=Nonethousands str | None=,encoding str | None=Nonedecimal str=.converters dict[typing.Any, typing.Any] | None=Nonena_values Iterable=Nonekeep_default_na bool=Truedisplayed_only bool=Trueextract_links Optional=Nonedtype_backend Literal=numpy_nullablestorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| stata |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderconvert_dates bool=Trueconvert_categoricals bool=Trueindex_col str | None=Noneconvert_missing bool=Falsepreserve_dtypes bool=Truecolumns collections.abc.Sequence | None=Noneorder_categoricals bool=Truechunksize int | None=Noneiterator bool=Falsecompression Union=inferstorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| feather |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadercolumns collections.abc.Sequence | None=Noneuse_threads bool=Truestorage_options dict[str, typing.Any] | None=Nonedtype_backend Literal=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| feather |
str | typing.BinaryIO | _io.BytesIO | pathlib.Path | bytescolumns list[str] | list[int] | None=Nonen_rows int | None=Noneuse_pyarrow bool=Falsememory_map bool=Truestorage_options dict[str, typing.Any] | None=Nonerow_count_name str | None=Nonerow_count_offset int=0rechunk bool=True |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| feather |
str | typing.BinaryIO | _io.BytesIO | pathlib.Path | bytescolumns list[str] | list[int] | None=Nonen_rows int | None=Noneuse_pyarrow bool=Falsememory_map bool=Truestorage_options dict[str, typing.Any] | None=Nonerow_count_name str | None=Nonerow_count_offset int=0rechunk bool=True |
LazyFrame |
hamilton.plugins.polars_lazyframe_extensions |
| orc |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadercolumns list[str] | None=Nonedtype_backend Literal=numpy_nullablefilesystem pyarrow._fs.FileSystem | fsspec.spec.AbstractFileSystem | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| excel |
str | pathlib.Path | _io.BytesIO | _io.BufferedReader=Nonesheet_name str | int | list[int | str] | None=0header int | collections.abc.Sequence | None=0names collections.abc.Sequence | None=Noneindex_col int | str | collections.abc.Sequence | None=Noneusecols int | str | collections.abc.Sequence | collections.abc.Callable[[str], bool] | None=Nonedtype Union=Noneengine Optional=Noneconverters dict[str, collections.abc.Callable] | dict[int, collections.abc.Callable] | None=Nonetrue_values collections.abc.Iterable | None=Nonefalse_values collections.abc.Iterable | None=Noneskiprows collections.abc.Sequence | int | collections.abc.Callable[[int], object] | None=Nonenrows int | None=Nonekeep_default_na bool=Truena_filter bool=Trueverbose bool=Falseparse_dates list[int | str] | dict[str, list[int | str]] | bool=Falsedate_format dict[collections.abc.Hashable, str] | str | None=Nonethousands str | None=Nonedecimal str=.comment str | None=Noneskipfooter int=0storage_options dict[str, typing.Any] | None=Nonedtype_backend Literal=numpy_nullableengine_kwargs dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| table |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadersep str | None=Nonedelimiter str | None=Noneheader int | collections.abc.Sequence | str | None=infernames collections.abc.Sequence | None=Noneindex_col int | str | collections.abc.Sequence | None=Noneusecols collections.abc.Sequence | None=Nonedtype Union=Noneengine Optional=Noneconverters dict[collections.abc.Hashable, collections.abc.Callable] | None=Nonetrue_values collections.abc.Iterable | None=Nonefalse_values collections.abc.Iterable | None=Noneskipinitialspace bool=Falseskiprows list[int] | int | list[collections.abc.Callable] | None=Noneskipfooter int=0nrows int | None=Nonena_values collections.abc.Hashable | collections.abc.Iterable | dict[collections.abc.Hashable, collections.abc.Iterable] | None=Nonekeep_default_na bool=Truena_filter bool=Trueverbose bool=Falseskip_blank_lines bool=Trueparse_dates list[int | str] | dict[str, list[int | str]] | bool=Falseinfer_datetime_format bool=Falsekeep_date_col bool=Falsedate_parser collections.abc.Callable | None=Nonedate_format str | None=Nonedayfirst bool=Falsecache_dates bool=Trueiterator bool=Falsechunksize int | None=Nonecompression str | dict=inferthousands str | None=Nonedecimal str=.lineterminator str | None=Nonequotechar str | None="quoting int=0doublequote bool=Trueescapechar str | None=Nonecomment str | None=Noneencoding str | None=Noneencoding_errors str | None=strictdialect str | None=Noneon_bad_lines Union=errordelim_whitespace bool=Falselow_memory bool=Truememory_map bool=Falsefloat_precision Optional=Nonestorage_options dict | None=Nonedtype_backend Literal=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| fwf |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadercolspecs str | list[tuple[int, int]] | tuple[int, int]=inferwidths list[int] | None=Noneinfer_nrows int=100dtype_backend Literal=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| spss |
str | pathlib.Pathusecols list[collections.abc.Hashable] | collections.abc.Callable[[str], bool] | None=Noneconvert_categoricals bool=Truedtype_backend Literal=numpy_nullable |
DataFrame |
hamilton.plugins.pandas_extensions |
| avro |
str | typing.TextIO | _io.BytesIO | pathlib.Path | typing.BinaryIO | bytescolumns list[int] | list[str] | None=Nonen_rows int | None=None |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| ndjson |
str | pathlib.Path | io.IOBase | bytesschema collections.abc.Mapping[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]] | collections.abc.Sequence[str | tuple[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]]]=Noneschema_overrides collections.abc.Mapping[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]] | collections.abc.Sequence[str | tuple[str, typing.Union[ForwardRef('DataTypeClass'), ForwardRef('DataType'), type[int], type[float], type[bool], type[str], type['date'], type['time'], type['datetime'], type['timedelta'], type[list[typing.Any]], type[tuple[typing.Any, ...]], type[bytes], type[object], type['Decimal'], type[None], NoneType]]]=None |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| database |
strconnection Unioniter_batches bool=Falsebatch_size int | None=Noneschema_overrides dict[str, typing.Any] | None=Noneinfer_schema_length int | None=Noneexecute_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| spreadsheet |
str | pathlib.Path | io.IOBase | bytessheet_id int | collections.abc.Sequence[int] | None=Nonesheet_name str | list[str] | tuple[str] | None=Noneengine Literal=xlsx2csvengine_options dict[str, typing.Any] | None=Noneread_options dict[str, typing.Any] | None=Noneschema_overrides dict[str, typing.Any] | None=Noneraise_if_empty bool=True |
DataFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| dlt |
DltResource |
DataFrame |
hamilton.plugins.dlt_extensions |
| mlflow |
str | None=Nonemode Literal=trackingrun_id str | None=Nonepath str | pathlib.Path=modelmodel_name str | None=Noneversion str | int | None=Noneversion_alias str | None=Noneflavor module | str | None=Nonemlflow_kwargs dict=None |
Any |
hamilton.plugins.mlflow_extensions |
Data Savers¶
key |
saver params |
types |
module |
|---|---|---|---|
| json |
str |
dictlist |
hamilton.io.default_data_loaders |
| json |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadercompression str=inferdate_format str=epochdate_unit str=msdefault_handler collections.abc.Callable[[typing.Any], str | float | bool | list | dict | None] | None=Nonedouble_precision int=10force_ascii bool=Trueindex bool | None=Noneindent int=0lines bool=Falsemode str=worient str | None=Nonestorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| json |
io.IOBase | str | pathlib.Path |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| json |
str | os.PathLike |
XGBModelBooster |
hamilton.plugins.xgboost_extensions |
| file |
strencoding str=utf-8 |
str |
hamilton.io.default_data_loaders |
| file |
pathlib.Path | str |
bytesBytesIO |
hamilton.io.default_data_loaders |
| file |
str | pathlib.Pathnum_iteration int | None=Nonestart_iteration int=0importance_type Literal=split |
LGBMModelBoosterCVBooster |
hamilton.plugins.lightgbm_extensions |
| pickle |
str |
object |
hamilton.io.default_data_loaders |
| pickle |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadercompression str | dict[str, typing.Any] | None=inferprotocol int=5storage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| memory | Any |
hamilton.io.default_data_loaders | |
| yaml |
str | pathlib.Path |
strintfloatbooldictlist |
hamilton.plugins.yaml_extensions |
| plt |
str | os.PathLike | typing.IOdpi float | str | None=Noneformat str | None=Nonemetadata dict | None=Nonebbox_inches str | matplotlib.transforms.Bbox | None=Nonepad_inches float | str | None=Nonefacecolor str | float | tuple | None=Noneedgecolor str | float | tuple | None=Nonebackend str | None=Noneorientation str | None=Nonepapertype str | None=Nonetransparent bool | None=Nonebbox_extra_artists list[matplotlib.artist.Artist] | None=Nonepil_kwargs dict | None=None |
Figure |
hamilton.plugins.matplotlib_extensions |
| npy |
str | pathlib.Path | typing.IOallow_pickle bool | None=Nonefix_imports bool | None=None |
ndarray |
hamilton.plugins.numpy_extensions |
| csv |
str | pathlib.Path | _io.BytesIO | _io.BufferedReadersep str | None=,na_rep str=float_format str | collections.abc.Callable | None=Nonecolumns collections.abc.Sequence | None=Noneheader bool | list[str] | None=Trueindex bool | None=Falseindex_label collections.abc.Hashable | collections.abc.Iterator[collections.abc.Hashable] | None=Nonemode str=wencoding str | None=Nonecompression Union=inferquoting int | None=Nonequotechar str | None="lineterminator str | None=Nonechunksize int | None=Nonedate_format str | None=Nonedoublequote bool=Trueescapechar str | None=Nonedecimal str=.errors str=strictstorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| csv |
_io.BytesIO | _io.TextIOWrapper | str | pathlib.Pathinclude_header bool=Trueseparator str=,line_terminator str=
quote_char str="batch_size int=1024datetime_format str=Nonedate_format str=Nonetime_format str=Nonefloat_precision int=Nonenull_value str=Nonequote_style type=None |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| parquet |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderengine Literal=autocompression str | None=snappyindex bool | None=Nonepartition_cols list[str] | None=Nonestorage_options dict[str, typing.Any] | None=Noneextra_kwargs dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| parquet |
_io.BytesIO | _io.TextIOWrapper | str | pathlib.Pathcompression Any=zstdcompression_level int=Nonestatistics bool=Falserow_group_size int=Noneuse_pyarrow bool=Falsepyarrow_options dict=None |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| sql |
strdb_connection Anychunksize int | None=Nonedtype Union=Noneif_exists str=failindex bool=Trueindex_label collections.abc.Hashable | collections.abc.Iterator[collections.abc.Hashable] | None=Nonemethod str | collections.abc.Callable | None=Noneschema str | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| xml |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderindex bool=Trueroot_name str=datarow_name str=rowna_rep str | None=Noneattr_cols list[str] | None=Noneelems_cols list[str] | None=Nonenamespaces dict[str, str] | None=Noneprefix str | None=Noneencoding str=utf-8xml_declaration bool=Truepretty_print bool=Trueparser str=lxmlstylesheet str | pathlib.Path | _io.BytesIO | _io.BufferedReader | None=Nonecompression str | dict[str, typing.Any] | None=inferstorage_options dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| html |
str | pathlib.Path | _io.StringIO | None=Nonecolumns list[str] | None=Nonecol_space str | int | list | dict | None=Noneheader bool | None=Trueindex bool | None=Truena_rep str | None=NaNformatters list | tuple | dict | None=Nonefloat_format str | None=Nonesparsify bool | None=Trueindex_names bool | None=Truejustify str=Nonemax_rows int | None=Nonemax_cols int | None=Noneshow_dimensions bool=Falsedecimal str=.bold_rows bool=Trueclasses str | list[str] | tuple | None=Noneescape bool | None=Truenotebook Literal=Falseborder int=Nonetable_id str | None=Nonerender_links bool=Falseencoding str | None=utf-8 |
DataFrame |
hamilton.plugins.pandas_extensions |
| stata |
str | pathlib.Path | _io.BufferedReader=Noneconvert_dates dict[collections.abc.Hashable, str] | None=Nonewrite_index bool=Truebyteorder str | None=Nonetime_stamp datetime.datetime | None=Nonedata_label str | None=Nonevariable_labels dict[collections.abc.Hashable, str] | None=Noneversion Literal=114convert_strl str | None=Nonecompression Union=inferstorage_options dict[str, typing.Any] | None=Nonevalue_labels dict[collections.abc.Hashable, str] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| feather |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderdest str | None=Nonecompression Literal=Nonecompression_level int | None=Nonechunksize int | None=Noneversion int | None=2 |
DataFrame |
hamilton.plugins.pandas_extensions |
| feather |
typing.BinaryIO | _io.BytesIO | str | pathlib.Path | None=Nonecompression type=uncompressed |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| orc |
str | pathlib.Path | _io.BytesIO | _io.BufferedReaderengine Literal=pyarrowindex bool | None=Noneengine_kwargs dict[str, typing.Any] | None=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| excel |
str | pathlib.Path | _io.BytesIOsheet_name str=Sheet1na_rep str=float_format str | None=Nonecolumns collections.abc.Sequence | None=Noneheader collections.abc.Sequence | bool=Trueindex bool=Trueindex_label collections.abc.Hashable | collections.abc.Iterator[collections.abc.Hashable] | None=Nonestartrow int=0startcol int=0engine Optional=Nonemerge_cells bool=Trueinf_rep str=inffreeze_panes tuple[int, int] | None=Nonestorage_options dict[str, typing.Any] | None=Noneengine_kwargs dict[str, typing.Any] | None=Nonemode Optional=wif_sheet_exists Optional=Nonedatetime_format str=Nonedate_format str=None |
DataFrame |
hamilton.plugins.pandas_extensions |
| avro |
_io.BytesIO | _io.TextIOWrapper | str | pathlib.Pathcompression Any=uncompressed |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| ndjson |
io.IOBase | str | pathlib.Path |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| database |
strconnection Unionif_table_exists Literal=failengine Literal=sqlalchemy |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| spreadsheet |
type | _io.BytesIO | pathlib.Path | strworksheet str | None=Noneposition tuple[int, int] | str=A1table_style str | dict[str, typing.Any] | None=Nonetable_name str | None=Nonecolumn_formats collections.abc.Mapping[str | tuple[str, ...], str | collections.abc.Mapping[str, str]] | None=Nonedtype_formats dict[polars.datatypes.classes.DataType | polars.datatypes.classes.DataTypeClass, str] | None=Noneconditional_formats collections.abc.Mapping[str | collections.abc.Collection[str], str | collections.abc.Mapping[str, typing.Any] | collections.abc.Sequence[str | collections.abc.Mapping[str, typing.Any]]] | None=Noneheader_format dict[str, typing.Any] | None=Nonecolumn_totals collections.abc.Mapping[typing.Union[ForwardRef('str'), ForwardRef('Selector'), tuple[typing.Union[ForwardRef('str'), ForwardRef('Selector')]]], str] | collections.abc.Sequence[str] | bool | None=Nonecolumn_widths collections.abc.Mapping[str, tuple[str, ...] | int] | int | None=Nonerow_totals collections.abc.Mapping[str, str | collections.abc.Collection[str]] | collections.abc.Collection[str] | bool | None=Nonerow_heights dict[int | tuple[int, ...], int] | int | None=Nonesparklines dict[str, collections.abc.Sequence[str] | dict[str, typing.Any]] | None=Noneformulas dict[str, str | dict[str, str]] | None=Nonefloat_precision int=3include_header bool=Trueautofilter bool=Trueautofit bool=Falsehidden_columns collections.abc.Sequence[str] | str | None=Nonehide_gridlines bool=Nonesheet_zoom int | None=Nonefreeze_panes str | tuple[int, int] | tuple[str, int, int] | tuple[int, int, int, int] | None=None |
DataFrameLazyFrame |
hamilton.plugins.polars_post_1_0_0_extensions |
| png |
str | os.PathLikedpi float=200format str=pngmetadata dict | None=Nonebbox_inches str=Nonepad_inches float=0.1backend str | None=Nonepapertype str=Nonetransparent bool=Nonebbox_extra_artists list | None=Nonepil_kwargs dict | None=None |
ConfusionMatrixDisplayDetCurveDisplayPrecisionRecallDisplayPredictionErrorDisplayRocCurveDisplayDecisionBoundaryDisplayLearningCurveDisplayPartialDependenceDisplayValidationCurveDisplayFigure |
hamilton.plugins.sklearn_plot_extensions |
| dlt |
Pipelinetable_name strprimary_key str | None=Nonewrite_disposition Optional=Nonecolumns collections.abc.Sequence[dlt.common.schema.typing.TColumnSchema] | None=Noneschema dlt.common.schema.schema.Schema | None=Noneloader_file_format Optional=None |
IterableDataFrameTableRecordBatch |
hamilton.plugins.dlt_extensions |
| mlflow |
str | pathlib.Path=modelregister_as str | None=Noneflavor str | module | None=Nonerun_id str | None=Nonemlflow_kwargs dict=None |
Any |
hamilton.plugins.mlflow_extensions |