List

Retrieves a list of jobs for a cluster.

TypeScript
Python

import { cloudApi, serviceClients, Session } from "@yandex-cloud/nodejs-sdk";

const ListJobsRequest = cloudApi.dataproc.manager_job_service.ListJobsRequest;

(async () => {
  const authToken = process.env["YC_OAUTH_TOKEN"];
  const session = new Session({ oauthToken: authToken });
  const client = session.client(serviceClients.JobServiceClient);

  const result = await client.list(
    ListJobsRequest.fromPartial({
      // clusterId: "clusterId",
      // pageSize: 0,
      // pageToken: "pageToken",
      // filter: "filter"
    })
  );
  console.log(result);
})();

import os
import grpc
import yandexcloud

from yandex.cloud.dataproc.manager.v1.job_service_pb2_grpc import JobServiceStub
from yandex.cloud.dataproc.manager.v1.job_service_pb2 import ListJobsRequest

token = os.getenv("YC_OAUTH_TOKEN")

sdk = yandexcloud.SDK(token=token)

service = sdk.client(JobServiceStub)
response = service.List(
    ListJobsRequest(
        # cluster_id = "clusterId",
        # page_size = 0,
        # page_token = "pageToken",
        # filter = "filter"
    )
)
print(response)

ListJobsRequest

`clusterId` : string

ID of the cluster to list jobs for.

`pageSize` : int64

The maximum number of results per page to return. If the number of available results is larger than page_size, the service returns a ListJobsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. Default value: 100.

`pageToken` : string

Page token. To get the next page of results, set page_token to the ListJobsResponse.next_page_token returned by a previous list request.

`filter` : string

A filter expression that filters jobs listed in the response.

The expression must specify:

The field name. Currently you can use filtering only on Job.name field.
An = operator.
The value in double quotes ("). Must be 3-63 characters long and match the regular expression [a-z][-a-z0-9]{1,61}[a-z0-9]. Example of a filter: name=my-job.

ListJobsResponse

`jobs` : Job

List of jobs for the specified cluster.

`nextPageToken` : string

Token for getting the next page of the list. If the number of results is greater than the specified ListJobsRequest.page_size, use next_page_token as the value for the ListJobsRequest.page_token parameter in the next list request.

Each subsequent page will have its own next_page_token to continue paging through the results.

Job

A Data Proc job. For details about the concept, see documentation.

Status

STATUS_UNSPECIFIED
PROVISIONING
Job is logged in the database and is waiting for the agent to run it.
PENDING
Job is acquired by the agent and is in the queue for execution.
RUNNING
Job is being run in the cluster.
ERROR
Job failed to finish the run properly.
DONE
Job is finished.
CANCELLED
Job is cancelled.
CANCELLING
Job is waiting for cancellation.

`id` : string

ID of the job. Generated at creation time.

`clusterId` : string

ID of the Data Proc cluster that the job belongs to.

`createdAt` : google.protobuf.Timestamp

Creation timestamp.

`startedAt` : google.protobuf.Timestamp

The time when the job was started.

`finishedAt` : google.protobuf.Timestamp

The time when the job was finished.

`name` : string

Name of the job, specified in the JobService.Create request.

`createdBy` : string

The id of the user who created the job

`status` : Status

Job status.

One of jobSpec

Specification for the job.

mapreduceJob : MapreduceJob
Specification for a MapReduce job.
sparkJob : SparkJob
Specification for a Spark job.
pysparkJob : PysparkJob
Specification for a PySpark job.
hiveJob : HiveJob
Specification for a Hive job.

`applicationInfo` : ApplicationInfo

Attributes of YARN application.

MapreduceJob

`args` : string

Optional arguments to pass to the driver.

`jarFileUris` : string

JAR file URIs to add to CLASSPATH of the Data Proc driver and each task.

`fileUris` : string

URIs of resource files to be copied to the working directory of Data Proc drivers and distributed Hadoop tasks.

`archiveUris` : string

URIs of archives to be extracted to the working directory of Data Proc drivers and tasks.

`properties` : string

Property names and values, used to configure Data Proc and MapReduce.

One of driver

mainJarFileUri : string
HCFS URI of the .jar file containing the driver class.
mainClass : string
The name of the driver class.

SparkJob

`args` : string

Optional arguments to pass to the driver.

`jarFileUris` : string

JAR file URIs to add to CLASSPATH of the Data Proc driver and each task.

`fileUris` : string

URIs of resource files to be copied to the working directory of Data Proc drivers and distributed Hadoop tasks.

`archiveUris` : string

URIs of archives to be extracted to the working directory of Data Proc drivers and tasks.

`properties` : string

Property names and values, used to configure Data Proc and Spark.

`mainJarFileUri` : string

The HCFS URI of the JAR file containing the main class for the job.

`mainClass` : string

The name of the driver class.

`packages` : string

List of maven coordinates of jars to include on the driver and executor classpaths.

`repositories` : string

List of additional remote repositories to search for the maven coordinates given with --packages.

`excludePackages` : string

List of groupId:artifactId, to exclude while resolving the dependencies provided in --packages to avoid dependency conflicts.

PysparkJob

`args` : string

Optional arguments to pass to the driver.

`jarFileUris` : string

JAR file URIs to add to CLASSPATH of the Data Proc driver and each task.

`fileUris` : string

URIs of resource files to be copied to the working directory of Data Proc drivers and distributed Hadoop tasks.

`archiveUris` : string

URIs of archives to be extracted to the working directory of Data Proc drivers and tasks.

`properties` : string

Property names and values, used to configure Data Proc and PySpark.

`mainPythonFileUri` : string

URI of the file with the driver code. Must be a .py file.

`pythonFileUris` : string

URIs of Python files to pass to the PySpark framework.

`packages` : string

List of maven coordinates of jars to include on the driver and executor classpaths.

`repositories` : string

List of additional remote repositories to search for the maven coordinates given with --packages.

`excludePackages` : string

List of groupId:artifactId, to exclude while resolving the dependencies provided in --packages to avoid dependency conflicts.

HiveJob

`properties` : string

Property names and values, used to configure Data Proc and Hive.

`continueOnFailure` : bool

Flag indicating whether a job should continue to run if a query fails.

`scriptVariables` : string

Query variables and their values.

`jarFileUris` : string

JAR file URIs to add to CLASSPATH of the Hive driver and each task.

One of queryType

queryFileUri : string
URI of the script with all the necessary Hive queries.
queryList : QueryList
List of Hive queries to be used in the job.

ApplicationInfo

`id` : string

ID of YARN application

`applicationAttempts` : ApplicationAttempt

YARN application attempts

QueryList

`queries` : string

List of Hive queries.

ApplicationAttempt

`id` : string

ID of YARN application attempt

`amContainerId` : string

ID of YARN Application Master container

List

ListJobsRequest​

clusterId : string​

pageSize : int64​

pageToken : string​

filter : string​

ListJobsResponse​

jobs : Job​

nextPageToken : string​

Job​

Status​

id : string​

clusterId : string​

createdAt : google.protobuf.Timestamp​

startedAt : google.protobuf.Timestamp​

finishedAt : google.protobuf.Timestamp​

name : string​

createdBy : string​

status : Status​

One of jobSpec​

mapreduceJob : MapreduceJob​

sparkJob : SparkJob​

pysparkJob : PysparkJob​

hiveJob : HiveJob​

applicationInfo : ApplicationInfo​

MapreduceJob​

args : string​

jarFileUris : string​

fileUris : string​

archiveUris : string​

properties : string​

One of driver​

mainJarFileUri : string​

mainClass : string​

SparkJob​

args : string​

jarFileUris : string​

fileUris : string​

archiveUris : string​

properties : string​

mainJarFileUri : string​

mainClass : string​

packages : string​

repositories : string​

excludePackages : string​

PysparkJob​

args : string​

jarFileUris : string​

fileUris : string​

archiveUris : string​

properties : string​

mainPythonFileUri : string​

pythonFileUris : string​

packages : string​

repositories : string​

excludePackages : string​

HiveJob​

properties : string​

continueOnFailure : bool​

scriptVariables : string​

jarFileUris : string​

One of queryType​

queryFileUri : string​

queryList : QueryList​

ApplicationInfo​

id : string​

applicationAttempts : ApplicationAttempt​

QueryList​

queries : string​

ApplicationAttempt​

id : string​

amContainerId : string​

ListJobsRequest

`clusterId` : string

`pageSize` : int64

`pageToken` : string

`filter` : string

ListJobsResponse

`jobs` : Job

`nextPageToken` : string

Job

Status

`id` : string

`clusterId` : string

`createdAt` : google.protobuf.Timestamp

`startedAt` : google.protobuf.Timestamp

`finishedAt` : google.protobuf.Timestamp

`name` : string

`createdBy` : string

`status` : Status

One of jobSpec

`mapreduceJob` : MapreduceJob

`sparkJob` : SparkJob

`pysparkJob` : PysparkJob

`hiveJob` : HiveJob

`applicationInfo` : ApplicationInfo

MapreduceJob

`args` : string

`jarFileUris` : string

`fileUris` : string

`archiveUris` : string

`properties` : string

One of driver

`mainJarFileUri` : string

`mainClass` : string

SparkJob

`args` : string

`jarFileUris` : string

`fileUris` : string

`archiveUris` : string

`properties` : string

`mainJarFileUri` : string

`mainClass` : string

`packages` : string

`repositories` : string

`excludePackages` : string

PysparkJob

`args` : string

`jarFileUris` : string

`fileUris` : string

`archiveUris` : string

`properties` : string

`mainPythonFileUri` : string

`pythonFileUris` : string

`packages` : string

`repositories` : string

`excludePackages` : string

HiveJob

`properties` : string

`continueOnFailure` : bool

`scriptVariables` : string

`jarFileUris` : string

One of queryType

`queryFileUri` : string

`queryList` : QueryList

ApplicationInfo

`id` : string

`applicationAttempts` : ApplicationAttempt

QueryList

`queries` : string

ApplicationAttempt

`id` : string

`amContainerId` : string