Metadata API Reference

api/service.proto

Kubeflow Metadata Service REST API

Version: v1alpha1

Default request content-types: application/json

Default response content-types: application/json

Schemes: http, https

Tags: MetadataService

application/json

  • 200 OK

A successful response.

apiListArtifactTypesResponse

POST /api/v1alpha1/artifact_types

Tags: MetadataService

application/json

ml_metadataArtifactType

application/json

  • 200 OK

A successful response.

apiCreateArtifactTypeResponse

DELETE /api/v1alpha1/artifact_types/{name}

Tags: MetadataService

name path string

application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}

Tags:

name ArtifactType names are of the form . path string

Uses default content-types: application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts

Tags:

name path string

Uses default content-types: application/json

  • 200 OK

A successful response.

DELETE /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags:

id path string
name path string

Uses default content-types: application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService

application/json

  • 200 OK

A successful response.

apiGetArtifactResponse

NOTE: The order of the following RPC methods affects the order of matching a particular HTTP path. So put a more specific path pattern before a generic one. For example, GET /api/v1alpha1/artifact_types/{parent}/artifacts should appear before GET /api/v1alpha1/artifact_types/{name} to be possibly matched.

POST /api/v1alpha1/artifact_types/{parent}/artifacts

Tags: MetadataService

application/json

The Artifact to create. Note that Artifact.type_id is ignored.

ml_metadataArtifact

parent Creates the specified artifact as an instance of ArtifactType with this fully qualified name. |parent| takes the form . path string

application/json

A successful response.

apiCreateArtifactResponse

GET /api/v1alpha1/artifacts

Tags: MetadataService

name query string

application/json

  • 200 OK

A successful response.

POST /api/v1alpha1/events

Tags:

Uses default content-types: application/json

Uses default content-types: application/json

  • 200 OK

A successful response.

List events based on an artifact or execution id.

GET /api/v1alpha1/events/artifacts/{name}

Tags: MetadataService

name path string

application/json

  • 200 OK

A successful response.

apiListEventsResponse

List events based on an artifact or execution id.

GET /api/v1alpha1/events/executions/{name}

Tags: MetadataService

name path string

application/json

  • 200 OK

A successful response.

apiListEventsResponse

GET /api/v1alpha1/execution_types

Tags: MetadataService

application/json

  • 200 OK

A successful response.

apiListExecutionTypesResponse

POST /api/v1alpha1/execution_types

Tags: MetadataService

application/json

ml_metadataExecutionType

application/json

  • 200 OK

A successful response.

apiCreateExecutionTypeResponse

DELETE /api/v1alpha1/execution_types/{name}

Tags: MetadataService

application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}

Tags:

name path string

Uses default content-types: application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions

Tags:

name path string

Uses default content-types: application/json

  • 200 OK

A successful response.

Tags: MetadataService

id path string
name path string

application/json

  • 200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions/{id}

Tags:

id path string
name Execution name is like . path string

Uses default content-types: application/json

  • 200 OK

A successful response.

POST /api/v1alpha1/execution_types/{parent}/executions

application/json

The Execution to create. Note that Execution.type_id is ignored.

ml_metadataExecution

application/json

  • 200 OK

A successful response.

apiCreateExecutionResponse

GET /api/v1alpha1/executions

Tags: MetadataService

name query string

application/json

  • 200 OK

A successful response.

apiListExecutionsResponse

apiCreateArtifactResponse: object

apiCreateArtifactTypeResponse: object

apiCreateExecutionResponse: object

apiCreateExecutionTypeResponse: object

apiGetArtifactResponse: object

apiGetArtifactTypeResponse: object

apiGetExecutionResponse: object

apiGetExecutionTypeResponse: object

apiListArtifactsResponse: object

apiListArtifactTypesResponse: object

apiListEventsResponse: object

  • artifacts: object

apiListExecutionsResponse: object

apiListExecutionTypesResponse: object

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

  • steps: object[]
  • A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

ml_metadataAnyArtifactStructType: object

Every ArtifactStruct is a member of this type.

ml_metadataArtifact: object

  • id: string (int64)
  • The id of the artifact.
  • type_id: string (int64)
  • The id of an ArtifactType. Type must be specified when an artifact is created, and it cannot be changed.
  • uri: string
  • The uniform resource identifier of the physical artifact. May be empty if there is no physical artifact.
  • properties: object
  • Properties of the artifact. Properties must be specified in the ArtifactType.

ml_metadataValue

  • custom_properties: object
  • User provided custom properties which are not defined by its type.

ml_metadataArtifactStructType: object

The type of an ArtifactStruct. An artifact struct type represents an infinite set of artifact structs. It can specify the input or output type of an ExecutionType. See the more specific types referenced in the message for more details.

  • simple:
  • intersection:
  • none:
  • tuple:

ml_metadataArtifactType: object

  • id: string (int64)
  • The id of the type. 1-1 relationship between type names and IDs.
  • name: string
  • The name of the type.
  • properties: object
  • The schema of the type. Properties are always optional in the artifact. Properties of an artifact type can be expanded but not contracted (i.e., you can add columns but not remove them).

ml_metadataPropertyType

ml_metadataDictArtifactStructType: object

  • properties: object
  • Underlying properties for the type.

ml_metadataArtifactStructType

  • none_type_not_required: boolean (boolean)
  • If true, then if properties["foo"] can be None, then that key is not required.
  • extra_properties_type:
  • Extra keys are allowed that are not specified in properties. These keys must have the type specified below. If this is not specified, then extra properties are not allowed.

ml_metadataEvent: object

An event represents a relationship between an artifact ID and an execution. There are four kinds of events, relating to both input and output, as well as declared versus undeclared. For example, the DECLARED_INPUT events are part of the signature of an execution. For example, consider: my_execution({"data":[3,7],"schema":8}) Where 3, 7, and 8 are artifact_ids. Assuming execution_id is 12, this becomes: {artifact_id:3, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":0}]}} {artifact_id:7, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":1}]}} {artifact_id:8, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"schema"}]}} The INPUT is an artifact actually read by the execution. The OUTPUT is an artifact actually written by the execution. The DECLARED_OUTPUT are the artifacts that are the "official" output. For example, the trainer may output multiple caches of the parameters (as OUTPUT objects), but then finally write the SavedModel as a DECLARED_OUTPUT. TODO(martinz): add a type for Event, similar to ArtifactType.

  • artifact_id: string (int64)
  • The artifact id is required for an event, and should refer to an existing artifact.
  • execution_id: string (int64)
  • The execution_id is required for an event, and should refer to an existing execution.
  • path:
  • The path in an artifact struct, or the name of an artifact.
  • milliseconds_since_epoch: string (int64)

ml_metadataEventType: string , x ∈ { UNKNOWN (default) , DECLARED_OUTPUT , DECLARED_INPUT , INPUT , OUTPUT }

Events distinguish between an artifact that is written by the execution (possibly as a cache), versus artifacts that are part of the declared output of the Execution. For more information on what DECLARED_ means, see the comment on the message.

ml_metadataExecution: object

  • id: string (int64)
  • The id of the execution.
  • type_id: string (int64)
  • The id of an ExecutionType. The ExecutionType must be specified and cannot be changed.
  • last_known_state:
  • The last known state of an execution.
  • properties: object
  • Properties of the Execution. Properties must be specified in the ExecutionType.

ml_metadataValue

  • custom_properties: object
  • User provided custom properties which are not defined by its type.

ml_metadataExecutionState: string , x ∈ { UNKNOWN (default) , NEW , RUNNING , COMPLETE , FAILED }

TODO(martinz): consider adding INVALIDATED as a state.

ml_metadataExecutionType: object

  • id: string (int64)
  • The id of the type. 1-1 relationship between type names and IDs.
  • name: string
  • The name of the type.
  • properties: object
  • The schema of the type. Properties are always optional in the execution.

ml_metadataPropertyType

  • input_type:
  • The ArtifactStructType of the input. For example: { "dict":{ "properties":{ "schema":{ "union_type":{ "none":{}, "simple":{…schema type…} }, }, "data":{ "simple":{…data_type…} } } } } That would be an optional schema field with a required data field.

ml_metadataIntersectionArtifactStructType: object

A member of this type must satisfy all constraints. This primarily useful not as an end-user type, but something calculated as an intermediate type in the system.

For example, suppose you have a method: def infer_my_input_type(a): # try to infer the input type of this method. use_in_method_x(a) # with input type x_input use_in_method_y(a) # with input type y_input

Given this information, you know that infer_my_input_type has type {"intersection":{"constraints":[x_input, y_input]}}.

IntersectionArtifactStructType intersection_type = {"constraints":[ {"dict":{"properties":{"schema":{"any":{}}}, "extra_properties":{"any":{}}}}, {"dict":{"properties":{"data":{"any":{}}}, "extra_properties":{"any":{}}}}]} Since the first constraint requires the dictionary to have a schema property, and the second constraint requires it to have a data property, this is equivalent to: ArtifactStructType other_type = {"dict":{"properties":{"schema":{"any":{}},"data":{"any":{}}}}, "extra_properties":{"any":{}}}

ml_metadataListArtifactStructType: object

Represents an ArtifactStruct list type with homogeneous elements.

  • element: ml_metadataArtifactStructType
  • Every entry in the list must be of this type. Note: if this type is Any, then the list can have arbitrary elements.

ml_metadataNoneArtifactStructType: object

ml_metadataPropertyType: string , x ∈ { UNKNOWN (default) , INT , DOUBLE , STRING }

TODO(martinz): consider moving this inside some message, to avoid having literals directly in apo package.

ml_metadataTupleArtifactStructType: object

An ordered list of heterogenous artifact structs. The length of the list is fixed. Each position in the list can have a different type.

ml_metadataUnionArtifactStructType: object

Represents a union of types.

  • candidates: object[]
  • An artifact struct matches this type if it matches any of the candidates. If candidates is empty, this is a bottom type (matches no artifacts).

ml_metadataArtifactStructType

A value in properties.

  • int_value: string (int64)
  • double_value: number (double)
  • string_value: string

PathStep: object

  • index: string (int64)
  • key: string