-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
What's the issue?
Summary
When running dagster-dbt integration with the DbtProjectComponent, the execution fails with AttributeError: 'str' object has no attribute 'joinpath' when dagster tries to attach SQL model code references.
Environment
- dagster: >=1.11.13
- dagster-dbt: >=0.28.1
- Python: 3.13
- dbt-core: >=1.10.13
- dbt-trino: >=1.9.3
Error Stacktrace
AttributeError: 'str' object has no attribute 'joinpath'
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/dagster_dbt_translator.py", line 725, in _attach_sql_model_code_reference
abs_path = project.project_dir.joinpath(relative_path).resolve()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^Full stack trace:
AttributeError: 'str' object has no attribute 'joinpath'
File "/var/lang/lib/python3.13/site-packages/dagster/_core/execution/plan/utils.py", line 57, in op_execution_error_boundary
yield
File "/var/lang/lib/python3.13/site-packages/dagster/_utils/__init__.py", line 394, in iterate_with_context
next_output = next(iterator)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/components/dbt_project/component.py", line 401, in _fn
yield from self.execute(context=context, dbt=DbtCliResource(project))
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/components/dbt_project/component.py", line 486, in execute
yield from self._get_dbt_event_iterator(context, dbt)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/core/dbt_event_iterator.py", line 215, in __next__
return next(self._inner_iterator)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/core/dbt_cli_invocation.py", line 289, in _stream_asset_events
yield from event.to_default_asset_events(
manifest=self.manifest,
dagster_dbt_translator=self.dagster_dbt_translator,
context=self.context,
target_path=self.target_path,
project=self.project,
)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/core/dbt_cli_event.py", line 570, in to_default_asset_events
yield from self._to_model_events(
manifest, dagster_dbt_translator, context, target_path, project
)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/core/dbt_cli_event.py", line 392, in _to_model_events
asset_key = dagster_dbt_translator.get_asset_spec(manifest, self._unique_id, project).key
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/components/dbt_project/component.py", line 532, in get_asset_spec
base_spec = super().get_asset_spec(manifest, unique_id, project)
File "/var/lang/lib/python3.13/site-packages/dagster/components/utils/translation.py", line 220, in get_asset_spec
return self._shim_method("get_asset_spec")(*args, **kwargs)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/dagster_dbt_translator.py", line 150, in get_asset_spec
asset=self.get_asset_spec(manifest, upstream_id, project).key,
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/components/dbt_project/component.py", line 532, in get_asset_spec
base_spec = super().get_asset_spec(manifest, unique_id, project)
File "/var/lang/lib/python3.13/site-packages/dagster/components/utils/translation.py", line 220, in get_asset_spec
return self._shim_method("get_asset_spec")(*args, **kwargs)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/dagster_dbt_translator.py", line 210, in get_asset_spec
metadata=_attach_sql_model_code_reference(
existing_metadata=spec.metadata,
dbt_resource_props=resource_props,
project=project,
)
File "/var/lang/lib/python3.13/site-packages/dagster_dbt/dagster_dbt_translator.py", line 725, in _attach_sql_model_code_reference
abs_path = project.project_dir.joinpath(relative_path).resolve()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
What did you expect to happen?
When DbtProject is retrieved from metadata, the project_dir field should be a Path object, not a string, so that methods like joinpath() can be called on it.
How to reproduce?
defs.yaml:
type: dagster_dbt.DbtProjectComponent
attributes:
project: '{{ project_root }}/path/to/dbt/project'
cli_args:
- run- Set up a dagster-dbt project with
DbtProjectComponent - Configure it to run a subset of dbt models
- Run the asset that depends on Python assets and dbt models
- The error occurs when dagster tries to attach code references to the dbt models
Dagster version
dagster, version 1.12.3
Deployment type
Local
Deployment details
No response
Additional information
Root Cause Analysis
The issue occurs when DbtProject is stored in and retrieved from Dagster asset metadata. Here's the flow:
-
Storage: In
dagster_dbt_translator.py(line 200), theDbtProjectobject is stored in asset metadata:**({DAGSTER_DBT_PROJECT_METADATA_KEY: project} if project else {})
-
Serialization: When Dagster serializes the metadata,
Pathobjects in theDbtProjectare converted to strings (standard Python/JSON serialization behavior). -
Retrieval: In
asset_utils.py(line 395), the project is retrieved from metadata:project = first_metadata.get(DAGSTER_DBT_PROJECT_METADATA_KEY)
-
Deserialized State: The retrieved
projectnow hasproject_diras a string instead of aPathobject, because metadata serialization doesn't preserve Python types. -
Error: When
_attach_sql_model_code_reference()tries to callproject.project_dir.joinpath(), it fails becauseproject_diris now a string.
The @record_custom decorator on DbtProject doesn't help here because the object is being serialized/deserialized through Dagster's metadata system, not through the record's own serialization.
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.