ListActive

Retrieves a list of jobs for Data Proc cluster.

TypeScript

import { cloudApi, serviceClients, Session } from "@yandex-cloud/nodejs-sdk";

const ListJobsRequest = cloudApi.dataproc.manager_job_service.ListJobsRequest;

(async () => {
  const authToken = process.env["YC_OAUTH_TOKEN"];
  const session = new Session({ oauthToken: authToken });
  const client = session.client(serviceClients.JobServiceClient);

  const result = await client.listActive(
    ListJobsRequest.fromPartial({
      // clusterId: "clusterId",
      // pageSize: 0,
      // pageToken: "pageToken",
      // filter: "filter"
    })
  );
  console.log(result);
})();

Python

import os
import grpc
import yandexcloud

from yandex.cloud.dataproc.manager.v1.job_service_pb2_grpc import JobServiceStub
from yandex.cloud.dataproc.manager.v1.job_service_pb2 import ListJobsRequest

token = os.getenv("YC_OAUTH_TOKEN")

sdk = yandexcloud.SDK(token=token)

service = sdk.client(JobServiceStub)
response = service.ListActive(
    ListJobsRequest(
        # cluster_id = "clusterId",
        # page_size = 0,
        # page_token = "pageToken",
        # filter = "filter"
    )
)
print(response)

ListJobsRequest

clusterId : string

Required. ID of the cluster to list Data Proc jobs of.

pageSize : int64

The maximum number of results per page that should be returned. If the number of available results is larger than page_size, the service returns a next_page_token that can be used to get the next page of results in subsequent ListJobs requests. Acceptable values are 0 to 1000, inclusive. Default value: 100.

pageToken : string

Page token. Set page_token to the next_page_token returned by a previous ListJobs request to get the next page of results.

filter : string

String that describes a display filter.

ListJobsResponse

jobs : Job

Requested list of Data Proc jobs.

nextPageToken : string

This token allows you to get the next page of results for ListJobs requests, if the number of results is larger than page_size specified in the request. To get the next page, specify the value of next_page_token as a value for the page_token parameter in the next ListClusters request. Subsequent ListClusters requests will have their own next_page_token to continue paging through the results.

Job

Data Proc job.

Status

• STATUS_UNSPECIFIED

• PROVISIONING

  Job created in metadb and is waiting agent to acquire.

• PENDING

  Job acquired by agent and is waiting for execution.

• RUNNING

  Job is running.

• ERROR

  Job failed.

• DONE

  Job finished.

• CANCELLED

  Job cancelled.

• CANCELLING

  Job is waiting for cancellation.

id : string

Unique ID of the Data Proc job. This ID is assigned by MDB in the process of creating Data Proc job.

clusterId : string

ID of the Data Proc cluster.

createdAt : google.protobuf.Timestamp

The time when the Data Proc job was created.

startedAt : google.protobuf.Timestamp

The time when the Data Proc job was started.

finishedAt : google.protobuf.Timestamp

The time when the Data Proc job was finished.

name : string

Name of the Data Proc job.

createdBy : string

The id of the user who created the job

status : Status

Status.

One of jobSpec

Job specification.

• mapreduceJob : MapreduceJob

• sparkJob : SparkJob

• pysparkJob : PysparkJob

• hiveJob : HiveJob

applicationInfo : ApplicationInfo

Attributes of YARN application.

MapreduceJob

args : string

Optional arguments to the driver.

jarFileUris : string

URIs of file to run.

fileUris : string

URIs of files to be copied to the working directory of Data Proc drivers and distributed tasks.

archiveUris : string

URIs of archives to be extracted in the working directory of Data Proc drivers and tasks.

properties : string

A mapping of property names to values, used to configure Data Proc.

One of driver

• mainJarFileUri : string

  The HCFS URI of the jar file containing the main class.

• mainClass : string

  The name of the driver's main class.

SparkJob

args : string

Optional arguments to the driver.

jarFileUris : string

Jar file URIs to add to the CLASSPATHs of the Data Proc driver and tasks.

fileUris : string

URIs of files to be copied to the working directory of Data Proc drivers and distributed tasks.

archiveUris : string

URIs of archives to be extracted in the working directory of Data Proc drivers and tasks.

properties : string

A mapping of property names to values, used to configure Data Proc.

mainJarFileUri : string

The HCFS URI of the jar file containing the main class.

mainClass : string

The name of the driver's main class.

packages : string

List of maven coordinates of jars to include on the driver and executor classpaths.

repositories : string

List of additional remote repositories to search for the maven coordinates given with --packages.

excludePackages : string

List of groupId:artifactId, to exclude while resolving the dependencies provided in --packages to avoid dependency conflicts.

PysparkJob

args : string

Optional arguments to the driver.

jarFileUris : string

Jar file URIs to add to the CLASSPATHs of the Data Proc driver and tasks.

fileUris : string

URIs of files to be copied to the working directory of Data Proc drivers and distributed tasks.

archiveUris : string

URIs of archives to be extracted in the working directory of Data Proc drivers and tasks.

properties : string

A mapping of property names to values, used to configure Data Proc.

mainPythonFileUri : string

URI of the main Python file to use as the driver. Must be a .py file.

pythonFileUris : string

URIs of Python files to pass to the PySpark framework.

packages : string

List of maven coordinates of jars to include on the driver and executor classpaths.

repositories : string

List of additional remote repositories to search for the maven coordinates given with --packages.

excludePackages : string

List of groupId:artifactId, to exclude while resolving the dependencies provided in --packages to avoid dependency conflicts.

HiveJob

properties : string

A mapping of property names to values, used to configure Hive.

continueOnFailure : bool

Whether to continue executing queries if a query fails.

scriptVariables : string

Mapping of query variable names to values.

jarFileUris : string

Jar file URIs to add to the CLASSPATHs of the Hive driver and tasks.

One of queryType

• queryFileUri : string

  URI of the script that contains Hive queries.

• queryList : QueryList

ApplicationInfo

id : string

ID of YARN application

applicationAttempts : ApplicationAttempt

YARN application attempts

QueryList

queries : string

ApplicationAttempt

id : string

ID of YARN application attempt

amContainerId : string

ID of YARN Application Master container

---