23/06/28 07:44:50 INFO DriverDaemon$: Started Log4j2 23/06/28 07:44:52 INFO DriverDaemon$: Current JVM Version 1.8.0_362 23/06/28 07:44:52 INFO DriverDaemon$: ========== driver starting up ========== 23/06/28 07:44:52 INFO DriverDaemon$: Java: Azul Systems, Inc. 1.8.0_362 23/06/28 07:44:52 INFO DriverDaemon$: OS: Linux/amd64 5.15.0-1038-azure 23/06/28 07:44:52 INFO DriverDaemon$: CWD: /databricks/driver 23/06/28 07:44:52 INFO DriverDaemon$: Mem: Max: 531.7G loaded GCs: PS Scavenge, PS MarkSweep 23/06/28 07:44:52 INFO DriverDaemon$: Logging multibyte characters: ✓ 23/06/28 07:44:52 INFO DriverDaemon$: 'publicFile.rolling.rewrite' appender in root logger: class org.apache.logging.log4j.core.appender.rewrite.RewriteAppender 23/06/28 07:44:52 INFO DriverDaemon$: == Modules: 23/06/28 07:44:53 INFO DriverDaemon$: Starting prometheus metrics log export timer 23/06/28 07:44:53 INFO DriverConf: Configured global ops config data source LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Load GOC flag data from LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Configured feature flag data source LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Load feature flag from LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Configured feature flag data source LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Load feature flag from LaunchDarkly 23/06/28 07:44:53 WARN DriverConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:44:53 INFO DriverConf: Configured global ops config data source LaunchDarkly 23/06/28 07:44:53 INFO DriverConf: Load GOC flag data from LaunchDarkly 23/06/28 07:44:53 INFO DriverDaemon$: Loaded JDBC drivers in 50 ms 23/06/28 07:44:53 INFO DriverDaemon$: Universe Git Hash: c5e92d170c47a4507f0149906672730fd23c0ff8 23/06/28 07:44:53 INFO DriverDaemon$: Spark Git Hash: f5b0c920e0e39d00d18c6b6ddc34baa7515844c3 23/06/28 07:44:53 INFO SparkConfUtils$: Customize spark config according to file /tmp/custom-spark.conf 23/06/28 07:44:53 WARN RunHelpers$: Missing tag isolation client: java.util.NoSuchElementException: key not found: TagDefinition(clientType,The client type for a request, used for isolating resources for the request.,DATA_LABEL_SYSTEM_NOT_SENSITIVE,false,false,List(),UsageLogRedactionConfig(List())) 23/06/28 07:44:53 INFO DatabricksILoop$: Creating throwaway interpreter 23/06/28 07:44:53 INFO MetastoreMonitor$: Internal internal metastore configured (config=DbMetastoreConfig{host=consolidated-australiaeast-prod-metastore-addl-1.mysql.database.azure.com, port=3306, dbName=organization8119356306712193, user=rlWPUjSu9F0ZHCW4@consolidated-australiaeast-prod-metastore-addl-1}) 23/06/28 07:44:53 INFO DataSourceFactory$: DataSource Jdbc URL: jdbc:mariadb://consolidated-australiaeast-prod-metastore-addl-1.mysql.database.azure.com:3306/organization8119356306712193?useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertificate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt 23/06/28 07:44:53 INFO DriverCorral: Creating the driver context 23/06/28 07:44:53 INFO DatabricksILoop$: Class Server Dir: /local_disk0/tmp/repl/spark-1758048110757902745-2be112e6-25cd-4f64-af80-bd74d906e728 23/06/28 07:44:53 INFO HikariDataSource: metastore-monitor - Starting... 23/06/28 07:44:53 INFO HikariDataSource: metastore-monitor - Start completed. 23/06/28 07:44:53 INFO SparkConfUtils$: Customize spark config according to file /tmp/custom-spark.conf 23/06/28 07:44:53 WARN SparkConf: The configuration key 'spark.akka.frameSize' has been deprecated as of Spark 1.6 and may be removed in the future. Please use the new key 'spark.rpc.message.maxSize' instead. 23/06/28 07:44:53 INFO SparkContext: Running Spark version 3.3.1 23/06/28 07:44:54 INFO ResourceUtils: ============================================================== 23/06/28 07:44:54 INFO ResourceUtils: No custom resources configured for spark.driver. 23/06/28 07:44:54 INFO ResourceUtils: ============================================================== 23/06/28 07:44:54 INFO SparkContext: Submitted application: Databricks Shell 23/06/28 07:44:54 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(memory -> name: memory, amount: 65536, script: , vendor: , offHeap -> name: offHeap, amount: 464612, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0) 23/06/28 07:44:54 INFO HikariDataSource: metastore-monitor - Shutdown initiated... 23/06/28 07:44:54 INFO ResourceProfile: Limiting resource is cpu 23/06/28 07:44:54 INFO ResourceProfileManager: Added ResourceProfile id: 0 23/06/28 07:44:54 INFO HikariDataSource: metastore-monitor - Shutdown completed. 23/06/28 07:44:54 INFO MetastoreMonitor: Metastore healthcheck successful (connection duration = 796 milliseconds) 23/06/28 07:44:54 INFO SecurityManager: Changing view acls to: root 23/06/28 07:44:54 INFO SecurityManager: Changing modify acls to: root 23/06/28 07:44:54 INFO SecurityManager: Changing view acls groups to: 23/06/28 07:44:54 INFO SecurityManager: Changing modify acls groups to: 23/06/28 07:44:54 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() 23/06/28 07:44:54 INFO Utils: Successfully started service 'sparkDriver' on port 37693. 23/06/28 07:44:54 INFO SparkEnv: Registering MapOutputTracker 23/06/28 07:44:54 INFO SparkEnv: Registering BlockManagerMaster 23/06/28 07:44:54 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 23/06/28 07:44:54 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up 23/06/28 07:44:54 INFO SparkEnv: Registering BlockManagerMasterHeartbeat 23/06/28 07:44:54 INFO DiskBlockManager: Created local directory at /local_disk0/blockmgr-955c90cc-d3e4-4dfe-8f5e-1346a6e25ac6 23/06/28 07:44:54 INFO MemoryStore: MemoryStore started with capacity 750.7 GiB 23/06/28 07:44:54 INFO SparkEnv: Registering OutputCommitCoordinator 23/06/28 07:44:54 INFO SparkContext: Spark configuration: eventLog.rolloverIntervalSeconds=900 libraryDownload.sleepIntervalSeconds=5 libraryDownload.timeoutSeconds=180 spark.akka.frameSize=256 spark.app.name=Databricks Shell spark.app.startTime=1687938293947 spark.cleaner.referenceTracking.blocking=false spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebappScimClient spark.databricks.automl.serviceEnabled=true spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceautocompute.MACClientImpl spark.databricks.cloudProvider=Azure spark.databricks.cloudfetch.hasRegionSupport=true spark.databricks.cloudfetch.requesterClassName=*********(redacted) spark.databricks.clusterSource=UI spark.databricks.clusterUsageTags.attribute_tag_budget= spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env= spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer= spark.databricks.clusterUsageTags.attribute_tag_dust_suite= spark.databricks.clusterUsageTags.attribute_tag_service= spark.databricks.clusterUsageTags.autoTerminationMinutes=120 spark.databricks.clusterUsageTags.azureSubscriptionId=f242a3b6-370c-4b2e-971d-a0399c53d6f0 spark.databricks.clusterUsageTags.cloudProvider=Azure spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"Vendor","value":"Databricks"},{"key":"Creator","value":"chanh.tran@tafenswtest.edu.au"},{"key":"ClusterName","value":"Databricks-Purview"},{"key":"ClusterId","value":"0622-000257-omphh17r"},{"key":"CostCentre","value":"CTP/180529-01"},{"key":"Environment","value":"Test"},{"key":"Owner","value":"Enterprise Data Services"},{"key":"ServiceName","value":"TRIP"},{"key":"DatabricksEnvironment","value":"workerenv-8119356306712193"}] spark.databricks.clusterUsageTags.clusterAvailability=ON_DEMAND_AZURE spark.databricks.clusterUsageTags.clusterCreator=Webapp spark.databricks.clusterUsageTags.clusterFirstOnDemand=1 spark.databricks.clusterUsageTags.clusterGeneration=25 spark.databricks.clusterUsageTags.clusterId=0622-000257-omphh17r spark.databricks.clusterUsageTags.clusterLastActivityTime=1687937434448 spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false spark.databricks.clusterUsageTags.clusterLogDestination= spark.databricks.clusterUsageTags.clusterMaxWorkers=8 spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT spark.databricks.clusterUsageTags.clusterMinWorkers=2 spark.databricks.clusterUsageTags.clusterName=Databricks-Purview spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false spark.databricks.clusterUsageTags.clusterNodeType=Standard_E96ds_v5 spark.databricks.clusterUsageTags.clusterNumCustomTags=0 spark.databricks.clusterUsageTags.clusterNumSshKeys=0 spark.databricks.clusterUsageTags.clusterOwnerOrgId=8119356306712193 spark.databricks.clusterUsageTags.clusterOwnerUserId=*********(redacted) spark.databricks.clusterUsageTags.clusterPinned=false spark.databricks.clusterUsageTags.clusterPythonVersion=3 spark.databricks.clusterUsageTags.clusterResourceClass=default spark.databricks.clusterUsageTags.clusterScalingType=autoscaling spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU spark.databricks.clusterUsageTags.clusterSpotBidMaxPrice=-1.0 spark.databricks.clusterUsageTags.clusterState=Restarting spark.databricks.clusterUsageTags.clusterStateMessage=Starting Spark spark.databricks.clusterUsageTags.clusterTargetWorkers=2 spark.databricks.clusterUsageTags.clusterUnityCatalogMode=CUSTOM spark.databricks.clusterUsageTags.clusterWorkers=2 spark.databricks.clusterUsageTags.containerType=LXC spark.databricks.clusterUsageTags.dataPlaneRegion=australiaeast spark.databricks.clusterUsageTags.driverContainerId=54143fa41c2a41a68bf7dc5182636b69 spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.60.24.15 spark.databricks.clusterUsageTags.driverInstanceId=90292f3c1b7d4c3eafe0f39f66343605 spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.60.28.15 spark.databricks.clusterUsageTags.driverNodeType=Standard_E96ds_v5 spark.databricks.clusterUsageTags.driverPublicDns=20.5.66.227 spark.databricks.clusterUsageTags.effectiveSparkVersion=12.1.x-scala2.12 spark.databricks.clusterUsageTags.enableCredentialPassthrough=*********(redacted) spark.databricks.clusterUsageTags.enableDfAcls=false spark.databricks.clusterUsageTags.enableElasticDisk=true spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=*********(redacted) spark.databricks.clusterUsageTags.enableJdbcAutoStart=true spark.databricks.clusterUsageTags.enableJobsAutostart=true spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false spark.databricks.clusterUsageTags.enableSqlAclsOnly=false spark.databricks.clusterUsageTags.hailEnabled=false spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8119356306712193 spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=vnet-injection spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false spark.databricks.clusterUsageTags.isIMv2Enabled=false spark.databricks.clusterUsageTags.isServicePrincipalCluster=false spark.databricks.clusterUsageTags.isSingleUserCluster=*********(redacted) spark.databricks.clusterUsageTags.managedResourceGroup=databricks-rg-ZEDBWBI02T-vrlfjwyjm5sky spark.databricks.clusterUsageTags.ngrokNpipEnabled=false spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=1 spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=1 spark.databricks.clusterUsageTags.orgId=8119356306712193 spark.databricks.clusterUsageTags.privateLinkEnabled=false spark.databricks.clusterUsageTags.region=australiaeast spark.databricks.clusterUsageTags.runtimeEngine=STANDARD spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=true spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false spark.databricks.clusterUsageTags.sparkMasterUrlType=*********(redacted) spark.databricks.clusterUsageTags.sparkVersion=12.1.x-scala2.12 spark.databricks.clusterUsageTags.userId=*********(redacted) spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=*********(redacted) spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=*********(redacted) spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=*********(redacted) spark.databricks.clusterUsageTags.userProvidedSparkVersion=*********(redacted) spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8119356306712193 spark.databricks.credential.aws.secretKey.redactor=*********(redacted) spark.databricks.credential.redactor=*********(redacted) spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=*********(redacted) spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=*********(redacted) spark.databricks.credential.scope.fs.impl=*********(redacted) spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=*********(redacted) spark.databricks.delta.logStore.crossCloud.fatal=true spark.databricks.delta.multiClusterWrites.enabled=true spark.databricks.delta.preview.enabled=true spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true spark.databricks.driverNfs.enabled=true spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs spark.databricks.driverNodeTypeId=Standard_E96ds_v5 spark.databricks.enablePublicDbfsFuse=false spark.databricks.eventLog.dir=eventlogs spark.databricks.eventLog.enabled=true spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DBCEventLoggingListener spark.databricks.io.directoryCommit.enableLogicalDelete=false spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.ManagedCatalogClientImpl spark.databricks.metrics.filesystem_io_metrics=true spark.databricks.mlflow.autologging.enabled=true spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datasources.SQLHadoopMapReduceCommitProtocol spark.databricks.passthrough.adls.gen2.tokenProviderClassName=*********(redacted) spark.databricks.passthrough.adls.tokenProviderClassName=*********(redacted) spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=*********(redacted) spark.databricks.passthrough.glue.executorServiceFactoryClassName=*********(redacted) spark.databricks.passthrough.oauth.refresher.impl=*********(redacted) spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.backend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory spark.databricks.passthrough.s3a.tokenProviderClassName=*********(redacted) spark.databricks.preemption.enabled=true spark.databricks.privateLinkEnabled=false spark.databricks.python.defaultPythonRepl=ipykernel spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy spark.databricks.repl.enableClassFileCleanup=true spark.databricks.secret.envVar.keys.toRedact=*********(redacted) spark.databricks.secret.sparkConf.keys.toRedact=*********(redacted) spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServerDBUtils spark.databricks.session.share=false spark.databricks.sparkContextId=1758048110757902745 spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperBridge spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLogStore spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStore spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=*********(redacted) spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=*********(redacted) spark.databricks.workerNodeTypeId=Standard_E96ds_v5 spark.databricks.workspaceUrl=*********(redacted) spark.databricks.wsfs.workspacePrivatePreview=true spark.databricks.wsfsPublicPreview=true spark.delta.sharing.profile.provider.class=*********(redacted) spark.driver.allowMultipleContexts=false spark.driver.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED spark.driver.host=10.60.24.15 spark.driver.maxResultSize=4g spark.driver.port=37693 spark.driver.tempDirectory=/local_disk0/tmp spark.eventLog.enabled=false spark.executor.extraClassPath=/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/* spark.executor.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djava.io.tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails -verbose:gc -Xss4m -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni -Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl -Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl -Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory -Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -Dscala.reflect.runtime.disable.typetag.cache=true -Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -Ddatabricks.serviceName=spark-executor-1 spark.executor.id=driver spark.executor.memory=65536m spark.executor.tempDirectory=/local_disk0/tmp spark.extraListeners=io.openlineage.spark.agent.OpenLineageSparkListener spark.files.fetchFailure.unRegisterOutputOnHost=true spark.files.overwrite=true spark.files.useFetchCache=false spark.hadoop.databricks.dbfs.client.version=v2 spark.hadoop.databricks.fs.perfMetrics.enable=true spark.hadoop.databricks.loki.fileSystemCache.enabled=true spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false spark.hadoop.databricks.s3.verifyBucketExists.enabled=false spark.hadoop.databricks.s3commit.client.sslTrustAll=false spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem spark.hadoop.fs.abfs.impl.disable.cache=true spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem spark.hadoop.fs.abfss.impl.disable.cache=true spark.hadoop.fs.adl.impl=com.databricks.adl.AdlFileSystem spark.hadoop.fs.adl.impl.disable.cache=true spark.hadoop.fs.azure.authorization.caching.enable=false spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheInvalidatorImpl spark.hadoop.fs.azure.skip.metrics=true spark.hadoop.fs.azure.user.agent.prefix=*********(redacted) spark.hadoop.fs.cpfs-abfss.impl=*********(redacted) spark.hadoop.fs.cpfs-abfss.impl.disable.cache=true spark.hadoop.fs.cpfs-adl.impl=*********(redacted) spark.hadoop.fs.cpfs-adl.impl.disable.cache=true spark.hadoop.fs.cpfs-s3.impl=*********(redacted) spark.hadoop.fs.cpfs-s3a.impl=*********(redacted) spark.hadoop.fs.cpfs-s3n.impl=*********(redacted) spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3 spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1 spark.hadoop.fs.fcfs-abfs.impl=*********(redacted) spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true spark.hadoop.fs.fcfs-abfss.impl=*********(redacted) spark.hadoop.fs.fcfs-abfss.impl.disable.cache=true spark.hadoop.fs.fcfs-s3.impl=*********(redacted) spark.hadoop.fs.fcfs-s3.impl.disable.cache=true spark.hadoop.fs.fcfs-s3a.impl=*********(redacted) spark.hadoop.fs.fcfs-s3a.impl.disable.cache=true spark.hadoop.fs.fcfs-s3n.impl=*********(redacted) spark.hadoop.fs.fcfs-s3n.impl.disable.cache=true spark.hadoop.fs.fcfs-wasb.impl=*********(redacted) spark.hadoop.fs.fcfs-wasb.impl.disable.cache=true spark.hadoop.fs.fcfs-wasbs.impl=*********(redacted) spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSystem spark.hadoop.fs.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3 spark.hadoop.fs.gs.impl.disable.cache=true spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216 spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem spark.hadoop.fs.s3.impl.disable.cache=true spark.hadoop.fs.s3a.assumed.role.credentials.provider=*********(redacted) spark.hadoop.fs.s3a.attempts.maximum=10 spark.hadoop.fs.s3a.block.size=67108864 spark.hadoop.fs.s3a.connection.maximum=200 spark.hadoop.fs.s3a.connection.timeout=50000 spark.hadoop.fs.s3a.fast.upload=true spark.hadoop.fs.s3a.fast.upload.active.blocks=32 spark.hadoop.fs.s3a.fast.upload.default=true spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem spark.hadoop.fs.s3a.impl.disable.cache=true spark.hadoop.fs.s3a.max.total.tasks=1000 spark.hadoop.fs.s3a.multipart.size=10485760 spark.hadoop.fs.s3a.multipart.threshold=104857600 spark.hadoop.fs.s3a.retry.interval=250ms spark.hadoop.fs.s3a.retry.limit=6 spark.hadoop.fs.s3a.retry.throttle.interval=500ms spark.hadoop.fs.s3a.threads.max=136 spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem spark.hadoop.fs.s3n.impl.disable.cache=true spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.PersonalStagingFileSystem spark.hadoop.fs.stage.impl.disable.cache=true spark.hadoop.fs.wasb.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem spark.hadoop.fs.wasb.impl.disable.cache=true spark.hadoop.fs.wasbs.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem spark.hadoop.fs.wasbs.impl.disable.cache=true spark.hadoop.hive.hmshandler.retry.attempts=10 spark.hadoop.hive.hmshandler.retry.interval=2000 spark.hadoop.hive.server2.enable.doAs=false spark.hadoop.hive.server2.idle.operation.timeout=7200000 spark.hadoop.hive.server2.idle.session.timeout=900000 spark.hadoop.hive.server2.keystore.password=*********(redacted) spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-keystore.jks spark.hadoop.hive.server2.session.check.interval=60000 spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false spark.hadoop.hive.server2.thrift.http.port=10000 spark.hadoop.hive.server2.transport.mode=http spark.hadoop.hive.server2.use.SSL=true spark.hadoop.hive.warehouse.subdir.inherit.perms=false spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.client.DirectOutputCommitter spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2 spark.hadoop.parquet.abfs.readahead.optimization.enabled=true spark.hadoop.parquet.block.size.row.check.max=10 spark.hadoop.parquet.block.size.row.check.min=10 spark.hadoop.parquet.filter.columnindex.enabled=false spark.hadoop.parquet.memory.pool.ratio=0.5 spark.hadoop.parquet.page.metadata.validation.enabled=true spark.hadoop.parquet.page.size.check.estimate=false spark.hadoop.parquet.page.verify-checksum.enabled=true spark.hadoop.parquet.page.write-checksum.enabled=true spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=false spark.hadoop.spark.driverproxy.customHeadersToProperties=*********(redacted) spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000 spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30 spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000 spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30 spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.DirectParquetOutputCommitter spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.data.client.MapReduceDirectOutputCommitter spark.home=/databricks/spark spark.logConf=true spark.master=spark://10.60.24.15:7077 spark.memory.offHeap.enabled=true spark.memory.offHeap.size=487180992512 spark.metrics.conf=/databricks/spark/conf/metrics.properties spark.openlineage.host=https://functionapptafetst266g.azurewebsites.net spark.openlineage.namespace=adb-8119356306712193.13.#0622-000257-omphh17r spark.openlineage.url.param.code=*********(redacted) spark.openlineage.version=v1 spark.r.backendConnectionTimeout=604800 spark.r.numRBackendThreads=1 spark.rdd.compress=true spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-1758048110757902745-2be112e6-25cd-4f64-af80-bd74d906e728 spark.rpc.message.maxSize=256 spark.scheduler.listenerbus.eventqueue.capacity=20000 spark.scheduler.mode=FAIR spark.serializer.objectStreamReset=100 spark.shuffle.manager=SORT spark.shuffle.memoryFraction=0.2 spark.shuffle.reduceLocality.enabled=false spark.shuffle.service.enabled=true spark.shuffle.service.port=4048 spark.sparklyr-backend.threads=1 spark.sparkr.use.daemon=false spark.speculation=false spark.speculation.multiplier=3 spark.speculation.quantile=0.9 spark.sql.allowMultipleContexts=false spark.sql.hive.convertCTAS=true spark.sql.hive.convertMetastoreParquet=true spark.sql.hive.metastore.jars=/databricks/databricks-hive/* spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.databricks,com.codahale,com.fasterxml.jackson,shaded.databricks spark.sql.hive.metastore.version=0.13.0 spark.sql.legacy.createHiveTableByDefault=false spark.sql.parquet.cacheMetadata=true spark.sql.parquet.compression.codec=snappy spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.DirectoryAtomicCommitProtocol spark.sql.sources.default=delta spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.DatabricksCheckpointFileManager spark.sql.streaming.stopTimeout=15s spark.sql.warehouse.dir=*********(redacted) spark.storage.blockManagerTimeoutIntervalMs=300000 spark.storage.memoryFraction=0.5 spark.streaming.driver.writeAheadLog.allowBatching=true spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true spark.task.reaper.enabled=true spark.task.reaper.killTimeout=60s spark.ui.port=40001 spark.ui.prometheus.enabled=true spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosDbfsReadinessCheckClient spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosIamRoleCheckClient spark.worker.cleanup.enabled=false 23/06/28 07:44:54 WARN MetricsSystem: Using default name SparkStatusTracker for source because neither spark.metrics.namespace nor spark.app.id is set. 23/06/28 07:44:54 INFO log: Logging initialized @8044ms to org.eclipse.jetty.util.log.Slf4jLog 23/06/28 07:44:54 INFO Server: jetty-9.4.46.v20220331; built: 2022-03-31T16:38:08.030Z; git: bc17a0369a11ecf40bb92c839b9ef0a8ac50ea18; jvm 1.8.0_362-b09 23/06/28 07:44:55 INFO Server: Started @8248ms 23/06/28 07:44:55 INFO AbstractConnector: Started ServerConnector@282d7dcf{HTTP/1.1, (http/1.1)}{10.60.24.15:40001} 23/06/28 07:44:55 INFO Utils: Successfully started service 'SparkUI' on port 40001. 23/06/28 07:44:55 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@63c163e1{/,null,AVAILABLE,@Spark} 23/06/28 07:44:55 INFO DLTDebugger: Registered DLTDebuggerEndpoint at endpoint dlt-debugger 23/06/28 07:44:55 INFO DriverPluginContainer: Initialized driver component for plugin org.apache.spark.debugger.DLTDebuggerSparkPlugin. 23/06/28 07:44:55 WARN FairSchedulableBuilder: Fair Scheduler configuration file not found so jobs will be scheduled in FIFO order. To use fair scheduling, configure pools in fairscheduler.xml or set spark.scheduler.allocation.file to a file that contains the configuration. 23/06/28 07:44:55 INFO FairSchedulableBuilder: Created default pool: default, schedulingMode: FIFO, minShare: 0, weight: 1 23/06/28 07:44:55 INFO DatabricksEdgeConfigs: serverlessEnabled : false 23/06/28 07:44:55 INFO DatabricksEdgeConfigs: perfPackEnabled : false 23/06/28 07:44:55 INFO DatabricksEdgeConfigs: classicSqlEnabled : false 23/06/28 07:44:55 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://10.60.24.15:7077... 23/06/28 07:44:55 INFO TransportClientFactory: Successfully created connection to /10.60.24.15:7077 after 56 ms (0 ms spent in bootstraps) 23/06/28 07:44:55 INFO StandaloneSchedulerBackend: Connected to Spark cluster with app ID app-20230628074455-0000 23/06/28 07:44:55 INFO TaskSchedulerImpl: Task preemption enabled. 23/06/28 07:44:55 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 40507. 23/06/28 07:44:55 INFO NettyBlockTransferService: Server created on 10.60.24.15:40507 23/06/28 07:44:55 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20230628074455-0000/0 on worker-20230628074453-10.60.24.10-46081 (10.60.24.10:46081) with 96 core(s) 23/06/28 07:44:55 INFO StandaloneSchedulerBackend: Granted executor ID app-20230628074455-0000/0 on hostPort 10.60.24.10:46081 with 96 core(s), 64.0 GiB RAM 23/06/28 07:44:55 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20230628074455-0000/1 on worker-20230628074453-10.60.24.13-44653 (10.60.24.13:44653) with 96 core(s) 23/06/28 07:44:55 INFO StandaloneSchedulerBackend: Granted executor ID app-20230628074455-0000/1 on hostPort 10.60.24.13:44653 with 96 core(s), 64.0 GiB RAM 23/06/28 07:44:55 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 23/06/28 07:44:55 INFO BlockManager: external shuffle service port = 4048 23/06/28 07:44:55 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.60.24.15, 40507, None) 23/06/28 07:44:55 INFO BlockManagerMasterEndpoint: Registering block manager 10.60.24.15:40507 with 750.7 GiB RAM, BlockManagerId(driver, 10.60.24.15, 40507, None) 23/06/28 07:44:55 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.60.24.15, 40507, None) 23/06/28 07:44:55 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 10.60.24.15, 40507, None) 23/06/28 07:44:55 INFO DatabricksUtils: Disabling Databricks event logging listener because spark.extraListeners does not contain the Databricks event logger class 23/06/28 07:44:55 INFO SparkContext: Registered listener io.openlineage.spark.agent.OpenLineageSparkListener 23/06/28 07:44:55 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20230628074455-0000/0 is now RUNNING 23/06/28 07:44:55 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20230628074455-0000/1 is now RUNNING 23/06/28 07:44:56 INFO DatabricksILoop$: Finished creating throwaway interpreter 23/06/28 07:44:56 INFO ContextHandler: Stopped o.e.j.s.ServletContextHandler@63c163e1{/,null,STOPPED,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@258291de{/jobs,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3ae91bcc{/jobs/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@59172d2{/jobs/job,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@54da736e{/jobs/job/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@23307c6c{/stages,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@7202009e{/stages/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@65b07b9{/stages/stage,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@387ef6a7{/stages/stage/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@7d086485{/stages/pool,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3b171fbd{/stages/pool/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@20dc763b{/storage,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@4fe8e318{/storage/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@62cf873a{/storage/rdd,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@294b045b{/storage/rdd/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@1ced1d14{/environment,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@5263f554{/environment/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@95958d9{/executors,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3dfaa1e4{/executors/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@44cab872{/executors/threadDump,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@2db75e32{/executors/threadDump/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@e353e1d{/executors/heapHistogram,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@d4dbb34{/executors/heapHistogram/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@5d10e2b6{/static,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@5bbf3869{/,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@1f5a1ad4{/api,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@471d6571{/metrics,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@77a1df4d{/jobs/job/kill,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@244c0fbe{/stages/stage/kill,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@634d56eb{/metrics/json,null,AVAILABLE,@Spark} 23/06/28 07:44:56 INFO StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0 23/06/28 07:44:56 INFO SparkContext: Loading Spark Service RPC Server. Classloader stack:List(com.databricks.backend.daemon.driver.ClassLoaders$MultiReplClassLoader@53d2f0ec, com.databricks.backend.daemon.driver.ClassLoaders$LibraryClassLoader@88fc9cb, sun.misc.Launcher$AppClassLoader@5e481248, sun.misc.Launcher$ExtClassLoader@56c9bbd8) 23/06/28 07:44:56 INFO SparkServiceRPCServer: Initializing Spark Service RPC Server. Classloader stack: List(com.databricks.backend.daemon.driver.ClassLoaders$MultiReplClassLoader@53d2f0ec, com.databricks.backend.daemon.driver.ClassLoaders$LibraryClassLoader@88fc9cb, sun.misc.Launcher$AppClassLoader@5e481248, sun.misc.Launcher$ExtClassLoader@56c9bbd8) 23/06/28 07:44:56 INFO SparkServiceRPCServer: Starting Spark Service RPC Server 23/06/28 07:44:56 INFO SparkServiceRPCServer: Starting Spark Service RPC Server. Classloader stack: List(com.databricks.backend.daemon.driver.ClassLoaders$MultiReplClassLoader@53d2f0ec, com.databricks.backend.daemon.driver.ClassLoaders$LibraryClassLoader@88fc9cb, sun.misc.Launcher$AppClassLoader@5e481248, sun.misc.Launcher$ExtClassLoader@56c9bbd8) 23/06/28 07:44:56 INFO Server: jetty-9.4.46.v20220331; built: 2022-03-31T16:38:08.030Z; git: bc17a0369a11ecf40bb92c839b9ef0a8ac50ea18; jvm 1.8.0_362-b09 23/06/28 07:44:56 INFO AbstractConnector: Started ServerConnector@5b697993{HTTP/1.1, (http/1.1)}{0.0.0.0:15001} 23/06/28 07:44:56 INFO Server: Started @9969ms 23/06/28 07:44:56 INFO DatabricksILoop$: Successfully registered spark metrics in Prometheus registry 23/06/28 07:44:56 INFO DatabricksILoop$: Successfully initialized SparkContext 23/06/28 07:44:57 INFO SharedState: Scheduler stats enabled. 23/06/28 07:44:58 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir. 23/06/28 07:44:58 INFO SharedState: Warehouse path is 'dbfs:/user/hive/warehouse'. 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@56e9126f{/storage/iocache,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@579cfcc4{/storage/iocache/json,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@48cebb78{/SQL,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@4ad57db1{/SQL/json,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@313a3af8{/SQL/execution,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@5471342{/SQL/execution/json,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@686ace1f{/static/sql,null,AVAILABLE,@Spark} 23/06/28 07:44:58 INFO AsyncEventQueue: Process of event SparkListenerApplicationStart(Databricks Shell,Some(app-20230628074455-0000),1687938293947,root,None,None,None) by listener OpenLineageSparkListener took 1.806795514s. 23/06/28 07:44:58 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:44:58 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:45:00 INFO DriverConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Load feature flag from LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Load feature flag from LaunchDarkly 23/06/28 07:45:00 WARN DriverConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:00 INFO DriverConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:00 INFO DriverConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.60.24.13:58548) with ID 1, ResourceProfileId 0 23/06/28 07:45:01 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.60.24.10:44288) with ID 0, ResourceProfileId 0 23/06/28 07:45:01 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:01 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:01 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO DatabricksMountsStore: Mount store initialization: Attempting to get the list of mounts from metadata manager of DBFS 23/06/28 07:45:01 INFO log: Logging initialized @14928ms to shaded.v9_4.org.eclipse.jetty.util.log.Slf4jLog 23/06/28 07:45:01 INFO DynamicRpcConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 WARN DynamicRpcConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:01 INFO DynamicRpcConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO DynamicRpcConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Load feature flag from LaunchDarkly 23/06/28 07:45:01 WARN FeatureFlagRegisterConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:01 INFO FeatureFlagRegisterConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:01 INFO TypeUtil: JVM Runtime does not support Modules 23/06/28 07:45:02 INFO BlockManagerMasterEndpoint: Registering block manager 10.60.24.13:44035 with 487.7 GiB RAM, BlockManagerId(1, 10.60.24.13, 44035, None) 23/06/28 07:45:02 INFO BlockManagerMasterEndpoint: Registering block manager 10.60.24.10:41041 with 487.7 GiB RAM, BlockManagerId(0, 10.60.24.10, 41041, None) 23/06/28 07:45:02 INFO DatabricksMountsStore: Mount store initialization: Received a list of 16 mounts accessible from metadata manager of DBFS 23/06/28 07:45:02 INFO DatabricksMountsStore: Updated mounts cache. Changes: List((+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-trusted)), (+,DbfsMountPoint(s3a://databricks-datasets-sydney/, /databricks-datasets)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/co_test_raw1)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/adls-raw)), (+,DbfsMountPoint(unsupported-access-mechanism-for-path--use-mlflow-client:/, /databricks/mlflow-tracking)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-external)), (+,DbfsMountPoint(wasbs://dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193, /databricks-results)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-lakehouse)), (+,DbfsMountPoint(unsupported-access-mechanism-for-path--use-mlflow-client:/, /databricks/mlflow-registry)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-stage)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-raw)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-raw/curms)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalakeraw)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/co_test_raw)), (+,DbfsMountPoint(wasbs://dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193, /)), (+,DbfsMountPoint(abfss://zedlsbi01t.dfs.core.windows.net/, /mnt/datalake-curated))) 23/06/28 07:45:02 INFO DatabricksFileSystemV2Factory: Creating wasbs file system for wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net 23/06/28 07:45:02 INFO AzureNativeFileSystemStore: URI scheme: wasbs, using https for connections 23/06/28 07:45:02 INFO NativeAzureFileSystem: Delete with limit configurations: deleteFileCountLimitEnabled=false, deleteFileCountLimit=-1 23/06/28 07:45:02 INFO DbfsHadoop3: Initialized DBFS with DBFSV2 as the delegate. 23/06/28 07:45:02 INFO HiveConf: Found configuration file file:/databricks/hive/conf/hive-site.xml 23/06/28 07:45:03 INFO SessionManager: HiveServer2: Background operation thread pool size: 100 23/06/28 07:45:03 INFO SessionManager: HiveServer2: Background operation thread wait queue size: 100 23/06/28 07:45:03 INFO SessionManager: HiveServer2: Background operation thread keepalive time: 10 seconds 23/06/28 07:45:03 INFO AbstractService: Service:OperationManager is inited. 23/06/28 07:45:03 INFO AbstractService: Service:SessionManager is inited. 23/06/28 07:45:03 INFO SparkSQLCLIService: Service: CLIService is inited. 23/06/28 07:45:03 INFO AbstractService: Service:ThriftHttpCLIService is inited. 23/06/28 07:45:03 INFO HiveThriftServer2: Service: HiveServer2 is inited. 23/06/28 07:45:03 INFO AbstractService: Service:OperationManager is started. 23/06/28 07:45:03 INFO AbstractService: Service:SessionManager is started. 23/06/28 07:45:03 INFO SparkSQLCLIService: Service: CLIService is started. 23/06/28 07:45:03 INFO AbstractService: Service:ThriftHttpCLIService is started. 23/06/28 07:45:03 INFO ThriftCLIService: HTTP Server SSL: adding excluded protocols: [SSLv2, SSLv3] 23/06/28 07:45:03 INFO ThriftCLIService: HTTP Server SSL: SslContextFactory.getExcludeProtocols = [SSL, SSLv2, SSLv2Hello, SSLv3] 23/06/28 07:45:03 INFO Server: jetty-9.4.46.v20220331; built: 2022-03-31T16:38:08.030Z; git: bc17a0369a11ecf40bb92c839b9ef0a8ac50ea18; jvm 1.8.0_362-b09 23/06/28 07:45:03 INFO session: DefaultSessionIdManager workerName=node0 23/06/28 07:45:03 INFO session: No SessionScavenger set, using defaults 23/06/28 07:45:03 INFO session: node0 Scavenging every 660000ms 23/06/28 07:45:03 WARN SecurityHandler: ServletContext@o.e.j.s.ServletContextHandler@4cef1ea6{/,null,STARTING} has uncovered http methods for path: /* 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@4cef1ea6{/,null,AVAILABLE} 23/06/28 07:45:03 INFO SslContextFactory: x509=X509@6ddd71cc(1,h=[az-australiaeast.workers.prod.ns.databricks.com],a=[],w=[]) for Server@6c720765[provider=null,keyStore=file:///databricks/keys/jetty-ssl-driver-keystore.jks,trustStore=null] 23/06/28 07:45:03 INFO AbstractConnector: Started ServerConnector@76ae9345{SSL, (ssl, http/1.1)}{0.0.0.0:10000} 23/06/28 07:45:03 INFO Server: Started @16635ms 23/06/28 07:45:03 INFO ThriftCLIService: Started ThriftHttpCLIService in https mode on port 10000 path=/cliservice/* with 5...500 worker threads 23/06/28 07:45:03 INFO AbstractService: Service:HiveServer2 is started. 23/06/28 07:45:03 INFO HiveThriftServer2: HiveThriftServer2 started 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@41b6ae51{/sqlserver,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@60b90fd3{/sqlserver/json,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@2eaaef66{/sqlserver/session,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3a8dd899{/sqlserver/session/json,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO LibraryResolutionManager: Preferred maven central mirror is configured to https://maven-central.storage-download.googleapis.com/maven2/ 23/06/28 07:45:03 WARN OutgoingDirectNotebookBufferRateLimiter$: No value specified for db-outgoing-buffer-throttler-burst. Using default: 100000000000 23/06/28 07:45:03 WARN OutgoingDirectNotebookBufferRateLimiter$: No value specified for db-outgoing-buffer-throttler-steady-rate. Using default: 6000000000 23/06/28 07:45:03 WARN OutgoingDirectNotebookBufferRateLimiter$: No value specified for db-outgoing-buffer-throttler-warning-interval-sec. Using default: 60 23/06/28 07:45:03 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@7addc5e4{/StreamingQuery,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@2b811d1c{/StreamingQuery/json,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@507c795e{/StreamingQuery/statistics,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@2533cbcc{/StreamingQuery/statistics/json,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@73430c86{/static/sql,null,AVAILABLE,@Spark} 23/06/28 07:45:03 INFO JettyServer$: Creating thread pool with name ... 23/06/28 07:45:03 INFO JettyServer$: Thread pool created 23/06/28 07:45:03 INFO JettyServer$: Creating thread pool with name ... 23/06/28 07:45:03 INFO JettyServer$: Thread pool created 23/06/28 07:45:03 INFO DriverDaemon: Starting driver daemon... 23/06/28 07:45:03 INFO SparkConfUtils$: Customize spark config according to file /tmp/custom-spark.conf 23/06/28 07:45:03 WARN SparkConf: The configuration key 'spark.akka.frameSize' has been deprecated as of Spark 1.6 and may be removed in the future. Please use the new key 'spark.rpc.message.maxSize' instead. 23/06/28 07:45:03 INFO DriverDaemon$: Attempting to run: 'set up ttyd daemon' 23/06/28 07:45:03 INFO DriverDaemon$: Attempting to run: 'Configuring RStudio daemon' 23/06/28 07:45:03 INFO DriverDaemon$: Resetting the default python executable 23/06/28 07:45:03 INFO Utils: resolved command to be run: List(virtualenv, /local_disk0/.ephemeral_nfs/cluster_libraries/python, -p, /databricks/python/bin/python, --no-download, --no-setuptools, --no-wheel) 23/06/28 07:45:04 INFO PythonEnvCloneHelper$: Created python virtualenv: /local_disk0/.ephemeral_nfs/cluster_libraries/python 23/06/28 07:45:04 INFO Utils: resolved command to be run: List(/databricks/python/bin/python, -c, import sys; dirs=[p for p in sys.path if 'package' in p]; print(' '.join(dirs))) 23/06/28 07:45:04 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, -c, from distutils.sysconfig import get_python_lib; print(get_python_lib())) 23/06/28 07:45:04 INFO PythonEnvCloneHelper$: Created sites.pth at /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/sites.pth 23/06/28 07:45:04 INFO ClusterWidePythonEnvManager: Registered /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages with the WatchService sun.nio.fs.LinuxWatchService$LinuxWatchKey@3396d2a6 23/06/28 07:45:04 INFO DriverDaemon$: Attempting to run: 'Update root virtualenv' 23/06/28 07:45:05 INFO DriverDaemon$: Finished updating /etc/environment 23/06/28 07:45:05 INFO DriverDaemon$$anon$1: Message out thread ready 23/06/28 07:45:05 INFO NetstatUtil$: Running netstat -lnpt 23/06/28 07:45:05 INFO NetstatUtil$: netstat -lnpt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 118/systemd-resolve tcp 0 0 0.0.0.0:8652 0.0.0.0:* LISTEN 1626/gmetad tcp 0 0 0.0.0.0:8649 0.0.0.0:* LISTEN 1617/gmond tcp 0 0 0.0.0.0:8651 0.0.0.0:* LISTEN 1626/gmetad tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 143/sshd: /usr/sbin tcp 0 0 0.0.0.0:7681 0.0.0.0:* LISTEN 2607/ttyd tcp6 0 0 10.60.24.15:7077 :::* LISTEN 1302/java tcp6 0 0 :::10000 :::* LISTEN 2036/java tcp6 0 0 :::15001 :::* LISTEN 2036/java tcp6 0 0 :::1021 :::* LISTEN 224/wsfs tcp6 0 0 :::1017 :::* LISTEN 224/wsfs tcp6 0 0 :::80 :::* LISTEN 1651/apache2 tcp6 0 0 :::22 :::* LISTEN 143/sshd: /usr/sbin tcp6 0 0 :::6060 :::* LISTEN 1060/java tcp6 0 0 :::7071 :::* LISTEN 1060/java tcp6 0 0 10.60.24.15:37693 :::* LISTEN 2036/java tcp6 0 0 10.60.24.15:40507 :::* LISTEN 2036/java tcp6 0 0 10.60.24.15:40000 :::* LISTEN 1302/java tcp6 0 0 10.60.24.15:40001 :::* LISTEN 2036/java 23/06/28 07:45:05 INFO Server: jetty-9.4.46.v20220331; built: 2022-03-31T16:38:08.030Z; git: bc17a0369a11ecf40bb92c839b9ef0a8ac50ea18; jvm 1.8.0_362-b09 23/06/28 07:45:05 INFO AbstractConnector: Started ServerConnector@1a2d02d8{HTTP/1.1, (http/1.1)}{0.0.0.0:6061} 23/06/28 07:45:05 INFO Server: Started @18542ms 23/06/28 07:45:05 INFO Server: jetty-9.4.46.v20220331; built: 2022-03-31T16:38:08.030Z; git: bc17a0369a11ecf40bb92c839b9ef0a8ac50ea18; jvm 1.8.0_362-b09 23/06/28 07:45:05 INFO SslContextFactory: x509=X509@46110b82(1,h=[az-australiaeast.workers.prod.ns.databricks.com],a=[],w=[]) for Server@6b732d6[provider=null,keyStore=null,trustStore=null] 23/06/28 07:45:05 WARN config: Weak cipher suite TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA enabled for Server@6b732d6[provider=null,keyStore=null,trustStore=null] 23/06/28 07:45:05 WARN config: Weak cipher suite TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA enabled for Server@6b732d6[provider=null,keyStore=null,trustStore=null] 23/06/28 07:45:05 INFO AbstractConnector: Started ServerConnector@45db8ba6{SSL, (ssl, http/1.1)}{0.0.0.0:6062} 23/06/28 07:45:05 INFO Server: Started @18604ms 23/06/28 07:45:05 INFO DriverDaemon: Started comm channel server 23/06/28 07:45:05 INFO DriverDaemon: Driver daemon started. 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Load feature flag from LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Configured feature flag data source LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Load feature flag from LaunchDarkly 23/06/28 07:45:05 WARN DynamicInfoServiceConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Configured global ops config data source LaunchDarkly 23/06/28 07:45:05 INFO DynamicInfoServiceConf: Load GOC flag data from LaunchDarkly 23/06/28 07:45:05 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:45:05 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:45:05 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:45:05 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:45:05 INFO DriverCorral: Loading the root classloader 23/06/28 07:45:05 INFO DriverCorral: Starting sql repl ReplId-3d0cd-cc8ed-2a27f-c 23/06/28 07:45:05 INFO DriverCorral: Starting sql repl ReplId-28040-c2e1d-7cbd7-1 23/06/28 07:45:05 INFO DriverCorral: Starting sql repl ReplId-6f5cf-31a6f-afedc-3 23/06/28 07:45:05 INFO SQLDriverWrapper: setupRepl:ReplId-28040-c2e1d-7cbd7-1: finished to load 23/06/28 07:45:05 INFO SQLDriverWrapper: setupRepl:ReplId-3d0cd-cc8ed-2a27f-c: finished to load 23/06/28 07:45:05 INFO SQLDriverWrapper: setupRepl:ReplId-6f5cf-31a6f-afedc-3: finished to load 23/06/28 07:45:05 INFO DriverCorral: Starting sql repl ReplId-77be5-cc404-546ff-2 23/06/28 07:45:05 INFO SQLDriverWrapper: setupRepl:ReplId-77be5-cc404-546ff-2: finished to load 23/06/28 07:45:05 INFO DriverCorral: Starting sql repl ReplId-5f205-01b81-4f849-9 23/06/28 07:45:05 INFO SQLDriverWrapper: setupRepl:ReplId-5f205-01b81-4f849-9: finished to load 23/06/28 07:45:05 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:45:05 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:45:05 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:45:05 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:45:05 INFO DriverCorral: Starting r repl ReplId-2d44e-df835-01882-7 23/06/28 07:45:06 INFO ROutputStreamHandler: Connection succeeded on port 35781 23/06/28 07:45:06 INFO ROutputStreamHandler: Connection succeeded on port 38887 23/06/28 07:45:06 INFO RDriverLocal: 1. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: object created with for ReplId-2d44e-df835-01882-7. 23/06/28 07:45:06 INFO RDriverLocal: 2. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: initializing ... 23/06/28 07:45:06 INFO RDriverLocal: 3. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: started RBackend thread on port 38637 23/06/28 07:45:06 INFO RDriverLocal: 4. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: waiting for SparkR to be installed ... 23/06/28 07:45:06 WARN JettyServer$Builder: Unexpected exception: com.databricks.backend.common.rpc.DriverMessages$ShouldUseAutoscalingInfo com.databricks.backend.common.rpc.DriverMessages$ShouldUseAutoscalingInfo at com.databricks.backend.daemon.driver.DriverCorral.com$databricks$backend$daemon$driver$DriverCorral$$handleRPCRequest(DriverCorral.scala:866) at com.databricks.backend.daemon.driver.DriverCorral$$anonfun$receive$1.applyOrElse(DriverCorral.scala:910) at com.databricks.backend.daemon.driver.DriverCorral$$anonfun$receive$1.applyOrElse(DriverCorral.scala:908) at com.databricks.rpc.ServerBackend.$anonfun$internalReceive$2(ServerBackend.scala:119) at com.databricks.rpc.ServerBackend$$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:146) at com.databricks.rpc.ServerBackend$$anonfun$commonReceive$1.applyOrElse(ServerBackend.scala:146) at com.databricks.rpc.ServerBackend.$anonfun$internalReceive$1(ServerBackend.scala:116) at com.databricks.logging.UsageLogging.$anonfun$recordOperation$1(UsageLogging.scala:554) at com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging.scala:649) at com.databricks.logging.UsageLogging.$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:670) at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:411) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:167) at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:409) at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:406) at com.databricks.rpc.ServerBackend.withAttributionContext(ServerBackend.scala:24) at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:454) at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:439) at com.databricks.rpc.ServerBackend.withAttributionTags(ServerBackend.scala:24) at com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scala:644) at com.databricks.logging.UsageLogging.recordOperationWithResultTags$(UsageLogging.scala:563) at com.databricks.rpc.ServerBackend.recordOperationWithResultTags(ServerBackend.scala:24) at com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:554) at com.databricks.logging.UsageLogging.recordOperation$(UsageLogging.scala:524) at com.databricks.rpc.ServerBackend.recordOperation(ServerBackend.scala:24) at com.databricks.rpc.ServerBackend.internalReceive(ServerBackend.scala:115) at com.databricks.rpc.JettyServer$RequestManager.$anonfun$handleRPC$2(JettyServer.scala:959) at scala.util.Try$.apply(Try.scala:213) at com.databricks.rpc.JettyServer$RequestManager.handleRPC(JettyServer.scala:959) at com.databricks.rpc.JettyServer$RequestManager.handleRequestAndRespond(JettyServer.scala:875) at com.databricks.rpc.JettyServer$RequestManager.$anonfun$handleHttp$2(JettyServer.scala:504) at com.databricks.rpc.JettyServer$RequestManager.$anonfun$handleHttp$2$adapted(JettyServer.scala:479) at com.databricks.logging.activity.ActivityContextFactory$.$anonfun$withActivityInternal$4(ActivityContextFactory.scala:378) at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:411) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:167) at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:409) at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:406) at com.databricks.logging.activity.ActivityContextFactory$.withAttributionContext(ActivityContextFactory.scala:60) at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:454) at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:439) at com.databricks.logging.activity.ActivityContextFactory$.withAttributionTags(ActivityContextFactory.scala:60) at com.databricks.logging.activity.ActivityContextFactory$.withActivityInternal(ActivityContextFactory.scala:378) at com.databricks.logging.activity.ActivityContextFactory$.withServiceRequestActivity(ActivityContextFactory.scala:171) at com.databricks.rpc.JettyServer$RequestManager.handleHttp(JettyServer.scala:479) at com.databricks.rpc.JettyServer$RequestManager.doPost(JettyServer.scala:376) at javax.servlet.http.HttpServlet.service(HttpServlet.java:523) at com.databricks.rpc.HttpServletWithPatch.service(HttpServletWithPatch.scala:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at com.databricks.rpc.InstrumentedQueuedThreadPool$$anon$1.run(InstrumentedQueuedThreadPool.scala:80) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750) 23/06/28 07:45:08 INFO Utils: resolved command to be run: WrappedArray(getconf, PAGESIZE) 23/06/28 07:45:13 INFO DriverCorral: [Thread 288] AttachLibraries - candidate libraries: List(com/microsoft/azure/azure-sqldb-spark#1.0.2.jar) 23/06/28 07:45:13 INFO LibraryResolutionManager: Resolving with preferred maven central mirror https://maven-central.storage-download.googleapis.com/maven2/ 23/06/28 07:45:14 INFO RDriverLocal$: SparkR installation completed. 23/06/28 07:45:14 INFO RDriverLocal: 5. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: launching R process ... 23/06/28 07:45:14 INFO RDriverLocal: 6. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: cgroup isolation disabled, not placing R process in REPL cgroup. 23/06/28 07:45:14 INFO RDriverLocal: 7. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: starting R process on port 1100 (attempt 1) ... 23/06/28 07:45:14 INFO RDriverLocal$: Debugging command for R process builder: SIMBASPARKINI=/etc/simba.sparkodbc.ini R_LIBS=/local_disk0/.ephemeral_nfs/envs/rEnv-28ea3f72-20d9-4fa9-952c-feaffbf204ef:/databricks/spark/R/lib:/local_disk0/.ephemeral_nfs/cluster_libraries/r LD_LIBRARY_PATH=/opt/simba/sparkodbc/lib/64/ SPARKR_BACKEND_CONNECTION_TIMEOUT=604800 DB_STREAM_BEACON_STRING_START=DATABRICKS_STREAM_START-ReplId-2d44e-df835-01882-7 DB_STDOUT_STREAM_PORT=35781 SPARKR_BACKEND_AUTH_SECRET=db6af7e4c7cdb37d8bf435591256f4339a25d9948b7940611a0c605af128e5cb DB_STREAM_BEACON_STRING_END=DATABRICKS_STREAM_END-ReplId-2d44e-df835-01882-7 EXISTING_SPARKR_BACKEND_PORT=38637 ODBCINI=/etc/odbc.ini DB_STDERR_STREAM_PORT=38887 /bin/bash /local_disk0/tmp/_startR.sh1008447982768095360resource.r /local_disk0/tmp/_rServeScript.r5669688556700133209resource.r 1100 None 23/06/28 07:45:14 INFO RDriverLocal: 8. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: setting up BufferedStreamThread with bufferSize: 1000. 23/06/28 07:45:15 INFO RDriverLocal: 9. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: R process started with RServe listening on port 1100. 23/06/28 07:45:15 INFO RDriverLocal: 10. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: starting interpreter to talk to R process ... 23/06/28 07:45:16 WARN SparkContext: Using an existing SparkContext; some configuration may not take effect. 23/06/28 07:45:16 INFO ROutputStreamHandler: Successfully connected to stdout in the RShell. 23/06/28 07:45:16 INFO ROutputStreamHandler: Successfully connected to stderr in the RShell. 23/06/28 07:45:16 INFO RDriverLocal: 11. RDriverLocal.15b47ce6-fd1f-44ad-99ba-bc43aac6c7a6: R interpreter is connected. 23/06/28 07:45:16 INFO RDriverWrapper: setupRepl:ReplId-2d44e-df835-01882-7: finished to load 23/06/28 07:45:45 INFO DriverCorral: [Thread 282] AttachLibraries - candidate libraries: List(com/microsoft/azure/azure-sqldb-spark#1.0.2.jar) 23/06/28 07:45:45 INFO DriverCorral: [Thread 282] AttachLibraries - new libraries to install (including resolved dependencies): List() 23/06/28 07:46:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:46:11 INFO LibraryResolutionManager: Library org/codehaus/plexus/plexus-utils#3.0.20.jar is resolved with message: org/codehaus/plexus/plexus-utils#1.5.15.jar has been evicted due to: [org/codehaus/plexus/plexus-utils#3.0.20.jar]. 23/06/28 07:46:11 INFO LibraryResolutionManager: Library org/codehaus/plexus/plexus-utils#3.0.20.jar is resolved with message: org/codehaus/plexus/plexus-utils#1.5.5.jar has been evicted due to: [org/codehaus/plexus/plexus-utils#3.0.20.jar]. 23/06/28 07:46:11 INFO SharedDriverContext: Register resolved dependencies: Map(com/microsoft/azure/azure-sqldb-spark#1.0.2.jar -> List(JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE))) 23/06/28 07:46:11 INFO DriverCorral: [Thread 288] AttachLibraries - new libraries to install (including resolved dependencies): List(com/microsoft/azure/azure-sqldb-spark#1.0.2.jar, JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE), JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE)) 23/06/28 07:46:11 INFO SharedDriverContext: [Thread 288] attachLibrariesToSpark com/microsoft/azure/azure-sqldb-spark#1.0.2.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE) as local file /local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar in 15 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar,,NONE) to local file /local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar with timestamp 1687938371719 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar with timestamp 1687938371732 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.google.code.gson_gson-2.2.4.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE) as local file /local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar,,NONE) to local file /local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar with timestamp 1687938371738 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar with timestamp 1687938371746 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_adal4j-1.2.0.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE) as local file /local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar,,NONE) to local file /local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar with timestamp 1687938371750 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar with timestamp 1687938371758 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-interpolation-1.11.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE) as local file /local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar,,NONE) to local file /local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar with timestamp 1687938371762 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar with timestamp 1687938371772 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.activation_activation-1.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE) as local file /local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar,,NONE) to local file /local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar with timestamp 1687938371776 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar with timestamp 1687938371785 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.slf4j_slf4j-api-1.7.5.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE) as local file /local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar,,NONE) to local file /local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar with timestamp 1687938371789 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar with timestamp 1687938371797 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.scalactic_scalactic_2.11-3.0.4.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE) as local file /local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar,,NONE) to local file /local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar with timestamp 1687938371801 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar with timestamp 1687938371809 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-lang3-3.4.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar with timestamp 1687938371812 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar with timestamp 1687938371820 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-repository-metadata-2.2.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE) as local file /local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar,,NONE) to local file /local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar with timestamp 1687938371824 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar with timestamp 1687938371832 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/junit_junit-3.8.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE) as local file /local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar in 0 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar,,NONE) to local file /local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar with timestamp 1687938371836 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar with timestamp 1687938371843 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.github.stephenc.jcip_jcip-annotations-1.0-1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE) as local file /local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar,,NONE) to local file /local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar with timestamp 1687938371847 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar with timestamp 1687938371855 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/net.minidev_json-smart-1.3.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE) as local file /local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar,,NONE) to local file /local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar with timestamp 1687938371859 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar with timestamp 1687938371866 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/classworlds_classworlds-1.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE) as local file /local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar,,NONE) to local file /local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar with timestamp 1687938371869 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar with timestamp 1687938371878 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-sec-dispatcher-1.4.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar with timestamp 1687938371881 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar with timestamp 1687938371889 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-registry-2.2.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE) as local file /local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar,,NONE) to local file /local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar with timestamp 1687938371892 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar with timestamp 1687938371900 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.sonatype.plexus_plexus-cipher-1.4.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE) as local file /local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar,,NONE) to local file /local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar with timestamp 1687938371904 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar with timestamp 1687938371912 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.sqlserver_mssql-jdbc-6.4.0.jre8.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar in 0 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar with timestamp 1687938371915 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar with timestamp 1687938371923 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-plugin-api-2.2.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE) as local file /local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar,,NONE) to local file /local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar with timestamp 1687938371927 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar with timestamp 1687938371935 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.commons_commons-collections4-4.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE) as local file /local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar,,NONE) to local file /local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar with timestamp 1687938371938 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar with timestamp 1687938371949 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/javax.mail_mail-1.4.7.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar with timestamp 1687938371952 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar with timestamp 1687938371960 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-profile-2.2.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE) as local file /local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar,,NONE) to local file /local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar with timestamp 1687938371964 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar with timestamp 1687938371972 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_nimbus-jose-jwt-9.31.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar with timestamp 1687938371975 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar with timestamp 1687938371983 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-2.2.1.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE) as local file /local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar,,NONE) to local file /local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar with timestamp 1687938371986 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar 23/06/28 07:46:11 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar with timestamp 1687938371994 23/06/28 07:46:11 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar to Spark 23/06/28 07:46:11 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/commons-codec_commons-codec-1.10.jar 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE) 23/06/28 07:46:11 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE) is downloaded 23/06/28 07:46:11 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE) as local file /local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar in 1 milliseconds 23/06/28 07:46:11 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar,,NONE) to local file /local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar 23/06/28 07:46:11 INFO SparkContext: Added file /local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar with timestamp 1687938371997 23/06/28 07:46:11 INFO Utils: Copying /local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar with timestamp 1687938372006 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/backport-util-concurrent_backport-util-concurrent-3.1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE) as local file /local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar,,NONE) to local file /local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar with timestamp 1687938372010 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar with timestamp 1687938372017 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.codehaus.plexus_plexus-container-default-1.0-alpha-9-stable-1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE) as local file /local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar,,NONE) to local file /local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar with timestamp 1687938372021 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar with timestamp 1687938372028 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.microsoft.azure_azure-sqldb-spark-1.0.2.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar with timestamp 1687938372032 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar with timestamp 1687938372040 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-artifact-manager-2.2.1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE) as local file /local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar in 0 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar,,NONE) to local file /local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar with timestamp 1687938372044 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar with timestamp 1687938372051 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.plugins_maven-gpg-plugin-1.6.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar in 0 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar with timestamp 1687938372054 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar with timestamp 1687938372062 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-settings-2.2.1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar with timestamp 1687938372065 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar with timestamp 1687938372074 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-model-2.2.1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE) as local file /local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar in 0 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar,,NONE) to local file /local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar with timestamp 1687938372077 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar with timestamp 1687938372086 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_lang-tag-1.7.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE) as local file /local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar in 0 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar,,NONE) to local file /local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar with timestamp 1687938372089 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar with timestamp 1687938372096 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven.wagon_wagon-provider-api-1.0-beta-6.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE) as local file /local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar,,NONE) to local file /local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar with timestamp 1687938372099 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar with timestamp 1687938372106 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/com.nimbusds_oauth2-oidc-sdk-5.18.1.jar 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloading a library that was not in the cache: JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE) 23/06/28 07:46:12 INFO LibraryDownloadManager: Attempt 1: wait until library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE) is downloaded 23/06/28 07:46:12 INFO LibraryDownloadManager: Downloaded library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE) as local file /local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar in 1 milliseconds 23/06/28 07:46:12 INFO SharedDriverContext: Successfully saved library JavaJarId(file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar,,NONE) to local file /local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar 23/06/28 07:46:12 INFO SparkContext: Added file /local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar with timestamp 1687938372109 23/06/28 07:46:12 INFO Utils: Copying /local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar to /local_disk0/spark-5fbcb007-d780-4cc7-b110-e0c5fc3817e0/userFiles-41eb89ef-19e0-4b02-84cc-848120dd8408/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar 23/06/28 07:46:12 INFO SparkContext: Added JAR /local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar with timestamp 1687938372117 23/06/28 07:46:12 INFO SharedDriverContext: Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar to Spark 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/org.apache.maven_maven-project-2.2.1.jar 23/06/28 07:46:12 INFO LibraryState: [Thread 288] Successfully attached library maven;azure-sqldb-spark;com.microsoft.azure;1.0.2;; 23/06/28 07:47:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:47:29 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:47:29 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:47:29 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:47:29 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:47:29 INFO DriverCorral: Starting python repl ReplId-6a727-6a976-cc8e8-d 23/06/28 07:47:30 INFO ClusterLoadMonitor: Added query with execution ID:0. Current active queries:1 23/06/28 07:47:30 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$ 23/06/28 07:47:31 INFO SecuredHiveExternalCatalog: creating hiveClient from java.lang.Throwable at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:79) at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:77) at org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog.scala:113) at org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$withClient$1(HiveExternalCatalog.scala:153) at com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressReporter.scala:364) at com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressReporter.scala:34) at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:152) at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:313) at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:298) at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:293) at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:60) at org.apache.spark.sql.hive.HiveSessionStateBuilder.$anonfun$resourceLoader$1(HiveSessionStateBuilder.scala:67) at org.apache.spark.sql.hive.HiveSessionResourceLoader.client$lzycompute(HiveSessionStateBuilder.scala:165) at org.apache.spark.sql.hive.HiveSessionResourceLoader.client(HiveSessionStateBuilder.scala:165) at org.apache.spark.sql.hive.HiveSessionResourceLoader.$anonfun$addJar$1(HiveSessionStateBuilder.scala:169) at org.apache.spark.sql.hive.HiveSessionResourceLoader.$anonfun$addJar$1$adapted(HiveSessionStateBuilder.scala:168) at scala.collection.immutable.List.foreach(List.scala:431) at org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:168) at org.apache.spark.sql.execution.command.AddJarsCommand.$anonfun$run$1(resources.scala:38) at org.apache.spark.sql.execution.command.AddJarsCommand.$anonfun$run$1$adapted(resources.scala:36) at scala.collection.immutable.Stream.foreach(Stream.scala:533) at org.apache.spark.sql.execution.command.AddJarsCommand.run(resources.scala:36) at org.apache.spark.sql.execution.command.ExecutedCommandExec.$anonfun$sideEffectResult$1(commands.scala:80) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:78) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:77) at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:89) at org.apache.spark.sql.execution.QueryExecution$$anonfun$$nestedInanonfun$eagerlyExecuteCommands$1$1.$anonfun$applyOrElse$2(QueryExecution.scala:235) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$8(SQLExecution.scala:245) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:424) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$1(SQLExecution.scala:190) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.execution.SQLExecution$.withCustomExecutionEnv(SQLExecution.scala:144) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:374) at org.apache.spark.sql.execution.QueryExecution$$anonfun$$nestedInanonfun$eagerlyExecuteCommands$1$1.$anonfun$applyOrElse$1(QueryExecution.scala:235) at org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$withMVTagsIfNecessary(QueryExecution.scala:220) at org.apache.spark.sql.execution.QueryExecution$$anonfun$$nestedInanonfun$eagerlyExecuteCommands$1$1.applyOrElse(QueryExecution.scala:233) at org.apache.spark.sql.execution.QueryExecution$$anonfun$$nestedInanonfun$eagerlyExecuteCommands$1$1.applyOrElse(QueryExecution.scala:226) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:519) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:106) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:519) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:31) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:298) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:294) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:31) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:495) at org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:226) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:354) at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:226) at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:180) at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:171) at org.apache.spark.sql.Dataset.(Dataset.scala:247) at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:116) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.SparkSession.$anonfun$withActiveAndFrameProfiler$1(SparkSession.scala:1045) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.SparkSession.withActiveAndFrameProfiler(SparkSession.scala:1045) at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:107) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:841) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:830) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:874) at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:695) at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$new$6(DriverLocal.scala:401) at org.apache.spark.SafeAddJarOrFile$.safe(SafeAddJarOrFile.scala:31) at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$new$5(DriverLocal.scala:401) at com.databricks.sql.acl.CheckPermissions$.trusted(CheckPermissions.scala:1825) at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$new$4(DriverLocal.scala:400) at com.databricks.unity.EmptyHandle$.runWith(UCSHandle.scala:124) at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$new$3(DriverLocal.scala:393) at scala.util.Using$.resource(Using.scala:269) at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$new$2(DriverLocal.scala:392) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at com.databricks.backend.daemon.driver.DriverLocal.(DriverLocal.scala:379) at com.databricks.backend.daemon.driver.PythonDriverLocalBase.(PythonDriverLocalBase.scala:168) at com.databricks.backend.daemon.driver.JupyterDriverLocal.(JupyterDriverLocal.scala:323) at com.databricks.backend.daemon.driver.PythonDriverWrapper.instantiateDriver(DriverWrapper.scala:760) at com.databricks.backend.daemon.driver.DriverWrapper.setupRepl(DriverWrapper.scala:349) at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:245) at java.lang.Thread.run(Thread.java:750) 23/06/28 07:47:31 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:47:31 INFO HiveUtils: Initializing HiveMetastoreConnection version 0.13.0 using file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--net.sf.jpam--jpam--net.sf.jpam__jpam__1.1.jar:file:/databricks/databricks-hive/----ws_3_3--mvn--hadoop3--org.apache.logging.log4j--log4j-slf4j-impl--org.apache.logging.log4j__log4j-slf4j-impl__2.18.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.thrift--libfb303--org.apache.thrift__libfb303__0.9.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.eclipse.jetty.aggregate--jetty-all--org.eclipse.jetty.aggregate__jetty-all__7.6.0.v20120127.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-ant--org.apache.hive__hive-ant__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-serde--org.apache.hive__hive-serde__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.zaxxer--HikariCP--com.zaxxer__HikariCP__2.5.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-lang--commons-lang--commons-lang__commons-lang__2.4.jar:file:/databricks/databricks-hive/manifest.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--antlr--antlr--antlr__antlr__2.7.7.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--stax--stax-api--stax__stax-api__1.0.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.mortbay.jetty--jetty--org.mortbay.jetty__jetty__6.1.26.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.zookeeper--zookeeper--org.apache.zookeeper__zookeeper__3.4.5.jar:file:/databricks/databricks-hive/----ws_3_3--mvn--hadoop3--org.slf4j--slf4j-api--org.slf4j__slf4j-api__1.7.36.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive.shims--hive-shims-common--org.apache.hive.shims__hive-shims-common__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.datanucleus--datanucleus-api-jdo--org.datanucleus__datanucleus-api-jdo__4.2.4.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-service--org.apache.hive__hive-service__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-common--org.apache.hive__hive-common__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.antlr--ST4--org.antlr__ST4__4.0.4.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--asm--asm-commons--asm__asm-commons__3.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.geronimo.specs--geronimo-jaspic_1.0_spec--org.apache.geronimo.specs__geronimo-jaspic_1.0_spec__1.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.servlet--servlet-api--javax.servlet__servlet-api__2.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.transaction--transaction-api--javax.transaction__transaction-api__1.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.avro--avro--org.apache.avro__avro__1.7.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.esotericsoftware.minlog--minlog--com.esotericsoftware.minlog__minlog__1.2.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-collections--commons-collections--commons-collections__commons-collections__3.2.2.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-io--commons-io--commons-io__commons-io__2.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-exec--org.apache.hive__hive-exec__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive.shims--hive-shims-0.20--org.apache.hive.shims__hive-shims-0.20__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.geronimo.specs--geronimo-annotation_1.0_spec--org.apache.geronimo.specs__geronimo-annotation_1.0_spec__1.1.1.jar:file:/databricks/databricks-hive/----ws_3_3--mvn--hadoop3--org.apache.logging.log4j--log4j-api--org.apache.logging.log4j__log4j-api__2.18.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive.shims--hive-shims-0.23--org.apache.hive.shims__hive-shims-0.23__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--junit--junit--junit__junit__3.8.1.jar:file:/databricks/databricks-hive/----ws_3_3--mvn--hadoop3--org.apache.logging.log4j--log4j-1.2-api--org.apache.logging.log4j__log4j-1.2-api__2.18.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.ant--ant-launcher--org.apache.ant__ant-launcher__1.9.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-jdbc--org.apache.hive__hive-jdbc__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--mvn--hadoop3--org.apache.logging.log4j--log4j-core--org.apache.logging.log4j__log4j-core__2.18.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-codec--commons-codec--commons-codec__commons-codec__1.8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.google.code.findbugs--jsr305--com.google.code.findbugs__jsr305__1.3.9.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.antlr--antlr-runtime--org.antlr__antlr-runtime__3.4.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.ant--ant--org.apache.ant__ant__1.9.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-shims--org.apache.hive__hive-shims__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.esotericsoftware.reflectasm--reflectasm-shaded--com.esotericsoftware.reflectasm__reflectasm-shaded__1.07.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.httpcomponents--httpcore--org.apache.httpcomponents__httpcore__4.2.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.httpcomponents--httpclient--org.apache.httpcomponents__httpclient__4.4.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.datanucleus--datanucleus-rdbms--org.datanucleus__datanucleus-rdbms__4.1.19.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.google.guava--guava--com.google.guava__guava__11.0.2.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-logging--commons-logging--commons-logging__commons-logging__1.1.3.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.codehaus.jackson--jackson-mapper-asl--org.codehaus.jackson__jackson-mapper-asl__1.9.13.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.commons--commons-compress--org.apache.commons__commons-compress__1.9.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.jolbox--bonecp--com.jolbox__bonecp__0.8.0.RELEASE.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.commons--commons-lang3--org.apache.commons__commons-lang3__3.4.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-beeline--org.apache.hive__hive-beeline__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.geronimo.specs--geronimo-jta_1.1_spec--org.apache.geronimo.specs__geronimo-jta_1.1_spec__1.1.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.jdo--jdo-api--javax.jdo__jdo-api__3.0.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.codehaus.jackson--jackson-core-asl--org.codehaus.jackson__jackson-core-asl__1.9.13.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-httpclient--commons-httpclient--commons-httpclient__commons-httpclient__3.0.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.antlr--stringtemplate--org.antlr__stringtemplate__3.2.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--commons-cli--commons-cli--commons-cli__commons-cli__1.2.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--oro--oro--oro__oro__2.0.8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.transaction--jta--javax.transaction__jta__1.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive.shims--hive-shims-0.20S--org.apache.hive.shims__hive-shims-0.20S__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.xerial.snappy--snappy-java--org.xerial.snappy__snappy-java__1.0.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.mortbay.jetty--servlet-api--org.mortbay.jetty__servlet-api__2.5-20081211.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.esotericsoftware.kryo--kryo--com.esotericsoftware.kryo__kryo__2.21.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.velocity--velocity--org.apache.velocity__velocity__1.5.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.datanucleus--datanucleus-core--org.datanucleus__datanucleus-core__4.1.17.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.mortbay.jetty--jetty-util--org.mortbay.jetty__jetty-util__6.1.26.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive.shims--hive-shims-common-secure--org.apache.hive.shims__hive-shims-common-secure__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.derby--derby--org.apache.derby__derby__10.10.1.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-cli--org.apache.hive__hive-cli__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--jline--jline--jline__jline__0.9.94.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.codehaus.groovy--groovy-all--org.codehaus.groovy__groovy-all__2.1.6.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--asm--asm-tree--asm__asm-tree__3.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--com.thoughtworks.paranamer--paranamer--com.thoughtworks.paranamer__paranamer__2.8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.ow2.asm--asm--org.ow2.asm__asm__4.0.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.mail--mail--javax.mail__mail__1.4.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.thrift--libthrift--org.apache.thrift__libthrift__0.9.2.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.datanucleus--javax.jdo--org.datanucleus__javax.jdo__3.2.0-m3.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--asm--asm--asm__asm__3.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.apache.hive--hive-metastore--org.apache.hive__hive-metastore__0.13.1-databricks-8.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--javax.activation--activation--javax.activation__activation__1.1.jar:file:/databricks/databricks-hive/----ws_3_3--maven-trees--hive-metastore-databricks-log4j2--org.objenesis--objenesis--org.objenesis__objenesis__1.2.jar:file:/databricks/databricks-hive/bonecp-configs.jar 23/06/28 07:47:31 INFO PoolingHiveClient: Hive metastore connection pool implementation is HikariCP 23/06/28 07:47:31 INFO LocalHiveClientsPool: Create Hive Metastore client pool of size 20 23/06/28 07:47:31 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:31 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:31 INFO HiveClientImpl: Warehouse location for Hive client (version 0.13.1) is dbfs:/user/hive/warehouse 23/06/28 07:47:31 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:47:31 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:47:31 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:47:31.289Z","run":{"runId":"e6995f55-6281-499d-a6fb-86625cd5cd5e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:31 INFO Persistence: Property datanucleus.fixedDatastore unknown - will be ignored 23/06/28 07:47:31 INFO Persistence: Property datanucleus.connectionPool.idleTimeout unknown - will be ignored 23/06/28 07:47:31 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 23/06/28 07:47:31 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 23/06/28 07:47:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:47:31 INFO HikariDataSource: HikariPool-1 - Started. 23/06/28 07:47:32 INFO HikariDataSource: HikariPool-2 - Started. 23/06/28 07:47:32 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 23/06/28 07:47:33 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:47:33 INFO HiveMetaStore: Added admin role in metastore 23/06/28 07:47:33 INFO HiveMetaStore: Added public role in metastore 23/06/28 07:47:33 INFO HiveMetaStore: No user is added in admin role, since config is empty 23/06/28 07:47:33 INFO HiveMetaStore: 0: get_database: default 23/06/28 07:47:33 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:0. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:1. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:1. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:2. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:2. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:3. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:3. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:4. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:4. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:5. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:5. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.498Z","run":{"runId":"e6995f55-6281-499d-a6fb-86625cd5cd5e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:6. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:6. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.528Z","run":{"runId":"29082be2-084d-49ef-a24e-699227cf5779","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:7. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:7. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:8. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:8. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.541Z","run":{"runId":"51c533e1-0394-4ca1-8855-d38a127fb0ce","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:9. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:9. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:10. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:10. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.554Z","run":{"runId":"a69154d4-9830-4394-b301-519890748d95","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:11. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:11. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:12. Current active queries:1 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:12. Current active queries:0 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.565Z","run":{"runId":"9f1d6715-a321-46c6-b67d-b1f6fc60a5fa","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:13. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:13. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:14. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:14. Current active queries:0 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.575Z","run":{"runId":"af408b3e-ebd1-42b9-809e-6a42e681f2ab","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:15. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:15. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:16. Current active queries:1 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.588Z","run":{"runId":"dec74449-d94b-48b1-a67d-76ec86121bc3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:16. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:17. Current active queries:1 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:17. Current active queries:0 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.599Z","run":{"runId":"ce4b0661-5180-4537-abb2-7b00986c24fa","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:18. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:18. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:19. Current active queries:1 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.609Z","run":{"runId":"f8db7626-abea-4a0b-948e-a8dd91a288d8","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:19. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:20. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:20. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:21. Current active queries:1 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.62Z","run":{"runId":"5d6434b0-8520-4e5c-8482-4ad1e5b2aa40","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:21. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:22. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:22. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:23. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:23. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:24. Current active queries:1 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.628Z","run":{"runId":"b6f6960e-f469-497d-a4a1-47004c804b12","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:24. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:25. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:25. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:26. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:26. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:27. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:27. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.64Z","run":{"runId":"3de0f5c0-9da2-41fd-be83-7f005006dcf5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:28. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:28. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:29. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:29. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:30. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:30. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:31. Current active queries:1 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:31. Current active queries:0 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.651Z","run":{"runId":"2a4cff60-8d8b-437e-995d-21b9d6f51b8b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:32. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:32. Current active queries:0 23/06/28 07:47:33 INFO ClusterLoadMonitor: Added query with execution ID:33. Current active queries:1 23/06/28 07:47:33 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:33 INFO ClusterLoadMonitor: Removed query with execution ID:33. Current active queries:0 23/06/28 07:47:33 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:33 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.658Z","run":{"runId":"41980b17-10fd-4774-a976-2c7865a00b5e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:33 INFO JupyterDriverLocal: Starting gateway server for repl ReplId-6a727-6a976-cc8e8-d 23/06/28 07:47:33 INFO PythonPy4JUtil: Using pinned thread mode in Py4J 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.667Z","run":{"runId":"f9ce155f-99da-4b4a-8d0b-b1897c55e4ad","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.68Z","run":{"runId":"014ce7c8-966a-4a4d-868f-95fc646c11e1","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.69Z","run":{"runId":"f4da367c-8b59-4a60-8d76-c722d9e776bf","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.699Z","run":{"runId":"933f0048-5425-47ca-8133-0a40fdb4fb4b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.711Z","run":{"runId":"237cb4c3-b033-41c8-bfee-142dea43d1fc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.722Z","run":{"runId":"04e2bb7c-6c58-4a48-8fac-92d43324b0d4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.733Z","run":{"runId":"8e0e07f9-1d5f-47e3-b210-5d3ac41e8dd9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.742Z","run":{"runId":"bb00a6d8-bce6-47c5-bddb-aacbb17cff6a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.749Z","run":{"runId":"ebdc99fa-a2f4-45d3-9e63-8ba528ba8edb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.756Z","run":{"runId":"8d697c89-beb0-4fe9-a008-a42c0016c982","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.765Z","run":{"runId":"66e1b218-fb87-491d-af72-db6acbaefc11","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.773Z","run":{"runId":"a91d91a3-3b60-49d0-b9b4-2679cc1162e9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.78Z","run":{"runId":"443ebae3-752c-4c38-a120-3f1e9f0c2029","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.787Z","run":{"runId":"a76003d4-eed1-4fad-9375-a28951532db7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.795Z","run":{"runId":"f2231528-64b7-4767-bb8a-a99a4e396908","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.802Z","run":{"runId":"d557d6d9-166d-4e4b-ac55-63f77d976ac6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.81Z","run":{"runId":"8d8512b0-0033-49d0-8a8f-727885e6598e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.818Z","run":{"runId":"858fa64e-57f6-4a43-810a-61de4a8bdd68","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.826Z","run":{"runId":"94c45a31-c0a4-48a7-853c-102975bb70e5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:34 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:33.832Z","run":{"runId":"841736b3-2bac-4837-a439-7a9252af30e2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:34 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:47:35 INFO VirtualenvCloneHelper: Creating notebook-scoped virtualenv for f212170d-1e72-46a5-82d8-d568a6994473 23/06/28 07:47:35 INFO Utils: resolved command to be run: List(virtualenv, /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473, -p, /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, --no-download, --no-setuptools, --no-wheel) 23/06/28 07:47:36 INFO PythonEnvCloneHelper$: Created python virtualenv: /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473 23/06/28 07:47:36 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, -c, import sys; dirs=[p for p in sys.path if 'package' in p]; print(' '.join(dirs))) 23/06/28 07:47:36 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473/bin/python, -c, from distutils.sysconfig import get_python_lib; print(get_python_lib())) 23/06/28 07:47:36 INFO PythonEnvCloneHelper$: Created sites.pth at /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473/lib/python3.9/site-packages/sites.pth 23/06/28 07:47:36 INFO NotebookScopedPythonEnvManager: Time spent to start virtualenv /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473 is 558(ms) 23/06/28 07:47:36 INFO NotebookScopedPythonEnvManager: Registered /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473/lib/python3.9/site-packages with the WatchService sun.nio.fs.LinuxWatchService$LinuxWatchKey@50c3db4c 23/06/28 07:47:36 INFO IpykernelUtils$: Python process builder: [bash, /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473/python_start_f212170d-1e72-46a5-82d8-d568a6994473.sh, /databricks/spark/python/pyspark/wrapped_python.py, root, /local_disk0/.ephemeral_nfs/envs/pythonEnv-f212170d-1e72-46a5-82d8-d568a6994473/bin/python, /databricks/python_shell/scripts/db_ipykernel_launcher.py, -f, /databricks/kernel-connections/8839c3b946eaeae1183a07109d853d3f3102a180bdcafd55f0cb843b27e1c267.json] 23/06/28 07:47:36 INFO IpykernelUtils$: Cgroup isolation disabled, not placing python process in repl cgroup 23/06/28 07:47:37 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:47:39 INFO PythonDriverWrapper: setupRepl:ReplId-6a727-6a976-cc8e8-d: finished to load 23/06/28 07:47:39 INFO DriverCorral: Starting scala repl ReplId-651be-d56dc-071dc-9 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:34. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:34. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:35. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:35. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:36. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:47:39.198Z","run":{"runId":"75147ed5-4f72-4956-b8dc-3e22d3183bec","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:36. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.201Z","run":{"runId":"75147ed5-4f72-4956-b8dc-3e22d3183bec","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:37. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:37. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:38. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.209Z","run":{"runId":"cda63ce6-1f08-40ce-8f98-cf3f17e1581f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:38. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:39. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:39. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:40. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.217Z","run":{"runId":"987e23dc-29cf-45c4-8af0-56c6e9830034","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:40. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:41. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.225Z","run":{"runId":"2c451c80-cb9a-429a-8d8b-9305145c316b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:41. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:42. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:42. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.231Z","run":{"runId":"387bd113-010f-44d0-bb7a-7963fed9d98d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:43. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:43. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:44. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:44. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.24Z","run":{"runId":"4fa1e94b-770b-489b-91d8-df1a66b43a90","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:45. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:45. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:46. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.246Z","run":{"runId":"80fa924e-ab5f-4033-a173-0c5e4931f91f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:46. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:47. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:47. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:48. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:48. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.255Z","run":{"runId":"7f77823c-7fe4-4db2-83f5-604dd7a55dcf","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:49. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:49. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:50. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.264Z","run":{"runId":"e6cf96d8-1869-4ef3-9e1c-add59eb0e304","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:50. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:51. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:51. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:52. Current active queries:1 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.271Z","run":{"runId":"66f102d3-1725-40ae-93c1-73948388aa24","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:52. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:53. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:53. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:54. Current active queries:1 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.277Z","run":{"runId":"a05358dd-80a8-4a7d-9c0f-4fff615f8af6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:54. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:55. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:55. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.283Z","run":{"runId":"a20f6899-bc7c-4bc5-a534-0effde2e2b3a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:56. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:56. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:57. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:57. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.288Z","run":{"runId":"f23738d6-ae47-4a31-8558-cdc67fe718d4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:58. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:58. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:59. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.294Z","run":{"runId":"f30b6a21-dbbb-4ac3-a723-c0b57c7781eb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:59. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:60. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:60. Current active queries:0 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:61. Current active queries:1 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.299Z","run":{"runId":"0818e5f1-c273-437d-9156-0e3a52036d32","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:61. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:62. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:62. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.305Z","run":{"runId":"9e89b1fb-da96-4ea3-b2b7-d9eebd70d925","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:63. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.311Z","run":{"runId":"71726bc0-a89c-4c0d-acff-99631497d523","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:63. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:64. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:64. Current active queries:0 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.316Z","run":{"runId":"cf0c2b62-c677-4250-af1f-efbd989550d9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:65. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:65. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:66. Current active queries:1 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.322Z","run":{"runId":"50837209-9536-448f-8485-05895b3811ff","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:66. Current active queries:0 23/06/28 07:47:39 INFO ClusterLoadMonitor: Added query with execution ID:67. Current active queries:1 23/06/28 07:47:39 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:47:39 INFO ClusterLoadMonitor: Removed query with execution ID:67. Current active queries:0 23/06/28 07:47:39 INFO ScalaDriverWrapper: setupRepl:ReplId-651be-d56dc-071dc-9: finished to load 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.327Z","run":{"runId":"1b864f82-4e7d-42a9-9a7f-858de0db1d35","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.333Z","run":{"runId":"e0f1c4a8-3473-4c92-995c-2206ef1b67f4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.338Z","run":{"runId":"4d0f7bee-43ef-4ada-bac7-f9a985633295","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.344Z","run":{"runId":"8a55623f-9277-40fd-a95d-44896d3510a9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.349Z","run":{"runId":"926bd093-4405-4cc3-be4c-f787f509f7c7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.354Z","run":{"runId":"3ecf1ba8-a3ce-4963-82a4-eabdc14a9b11","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.36Z","run":{"runId":"19130968-3856-4b6d-913a-0bbacc560a65","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.365Z","run":{"runId":"4e8f47ff-1549-4ae5-ab29-07b7bf350753","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.371Z","run":{"runId":"ff070cb3-a9cb-447d-908c-9e2e54f484ab","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.378Z","run":{"runId":"e12cae5c-9398-411c-af97-91b35be17cfb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.389Z","run":{"runId":"90b27d89-31ad-4751-ab64-3b25e6da0423","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.395Z","run":{"runId":"b199a7fa-e9aa-4abb-8f91-14954fa43e68","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.401Z","run":{"runId":"4f0494e8-2cfd-405e-85ff-865817f3ca14","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.407Z","run":{"runId":"8dc02ae7-b637-4f49-93db-5794b8748637","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:47:39 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:47:39.412Z","run":{"runId":"4ee72c17-501c-42a4-a6c6-7f47c0e231eb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:47:39 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8773623097537433174_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:39 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8773623097537433174_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8773623097537433174_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8427681459288369000_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8427681459288369000_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8427681459288369000_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6373175820824238908_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6373175820824238908_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6373175820824238908_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8972045534381232397_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8972045534381232397_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8972045534381232397_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9090892639600901207_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9090892639600901207_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9090892639600901207_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9083109230735962546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9083109230735962546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9083109230735962546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8847919232893772343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8847919232893772343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8847919232893772343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6553736259882943636_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6553736259882943636_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInRead' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInRead' instead. 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6553736259882943636_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6599105709514037972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6599105709514037972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInWrite' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInWrite' instead. 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6599105709514037972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6968063951588476247_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6968063951588476247_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6968063951588476247_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8427023413384964766_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:40 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8427023413384964766_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:41 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8427023413384964766_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:41 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8866768742800652501_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:41 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_8866768742800652501_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:41 INFO SignalUtils: Registering signal handler for INT 23/06/28 07:47:42 INFO DriverILoop: Set class prefix to: $line55d90a26dea8465bad6756e123631c04 23/06/28 07:47:42 INFO DriverILoop: set ContextClassLoader 23/06/28 07:47:42 INFO DriverILoop: initialized intp 23/06/28 07:47:44 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8866768742800652501_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5856220370601786891_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5856220370601786891_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5856220370601786891_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8328034455387764040_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8328034455387764040_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8328034455387764040_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5101040220490627328_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5101040220490627328_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5101040220490627328_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6631838744624702721_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6631838744624702721_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6631838744624702721_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6599003401901588227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6599003401901588227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6599003401901588227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8720304312733596001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8720304312733596001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8720304312733596001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5667301785278198796_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5667301785278198796_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5667301785278198796_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6933941116463880609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6933941116463880609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6933941116463880609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6093424168427069792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6093424168427069792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6093424168427069792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8844851066760476375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8844851066760476375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:45 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8844851066760476375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5855482950692954561_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5855482950692954561_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5855482950692954561_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7618894663123943382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7618894663123943382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7618894663123943382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8339299744019528543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8339299744019528543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8339299744019528543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6813157912479809424_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6813157912479809424_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6813157912479809424_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8903083418840740459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_8903083418840740459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8903083418840740459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4840086037330325987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4840086037330325987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4840086037330325987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8299454383843296575_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8299454383843296575_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8299454383843296575_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7059919491304342025_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7059919491304342025_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7059919491304342025_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5548150598148417497_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5548150598148417497_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5548150598148417497_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8141109788230619557_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:46 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8141109788230619557_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8141109788230619557_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6204587503705556347_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6204587503705556347_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6204587503705556347_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6442205314230526197_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6442205314230526197_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6442205314230526197_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6528042423646825288_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6528042423646825288_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6528042423646825288_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4977567206083051316_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4977567206083051316_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4977567206083051316_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5686076941774732625_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5686076941774732625_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5686076941774732625_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9201301272133634252_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9201301272133634252_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9201301272133634252_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5833403762903325933_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5833403762903325933_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5833403762903325933_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8685439730662463405_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8685439730662463405_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8685439730662463405_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8932382536511152160_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8932382536511152160_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8932382536511152160_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7046189816318438881_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:47 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7046189816318438881_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7046189816318438881_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5651206522420695012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5651206522420695012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5651206522420695012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4699043957176755653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4699043957176755653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4699043957176755653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6589376968603129698_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6589376968603129698_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6589376968603129698_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4708402577576879121_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4708402577576879121_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4708402577576879121_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4860683871788763621_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4860683871788763621_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4860683871788763621_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7719105669966203677_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7719105669966203677_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7719105669966203677_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7301751300258237802_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7301751300258237802_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7301751300258237802_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5718057065245335030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5718057065245335030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5718057065245335030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7770389032534436109_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7770389032534436109_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7770389032534436109_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5136943792275510473_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:48 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5136943792275510473_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5136943792275510473_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5336492106679338360_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5336492106679338360_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5336492106679338360_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4739369048955732107_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4739369048955732107_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4739369048955732107_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7391474017733988546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7391474017733988546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7391474017733988546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7822847185454284049_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7822847185454284049_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7822847185454284049_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5948495970637855310_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5948495970637855310_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5948495970637855310_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8288472644982568991_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8288472644982568991_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8288472644982568991_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6729565007423836361_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6729565007423836361_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6729565007423836361_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4927160904903609411_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4927160904903609411_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4927160904903609411_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5328977789968174525_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5328977789968174525_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5328977789968174525_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6303462749126302792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:49 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6303462749126302792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6303462749126302792_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6817678217821439603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6817678217821439603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6817678217821439603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9043035527466023330_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9043035527466023330_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9043035527466023330_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7514598683944864653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7514598683944864653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7514598683944864653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5972864639360657849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5972864639360657849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5972864639360657849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5972437612332493322_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5972437612332493322_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5972437612332493322_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4932009549601892777_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4932009549601892777_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4932009549601892777_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7483682657114480840_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7483682657114480840_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7483682657114480840_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6575197850169219531_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6575197850169219531_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6575197850169219531_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6765343475397272462_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6765343475397272462_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6765343475397272462_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5641462103049851844_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:50 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5641462103049851844_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5641462103049851844_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7254493008811791321_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7254493008811791321_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7254493008811791321_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5535718155416105093_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5535718155416105093_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5535718155416105093_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5659083631176705828_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5659083631176705828_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5659083631176705828_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7076151764562535035_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7076151764562535035_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7076151764562535035_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8410718700221876339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8410718700221876339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8410718700221876339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7866207635522036829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7866207635522036829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7866207635522036829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4892387232595171439_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4892387232595171439_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4892387232595171439_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8597227766525100338_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8597227766525100338_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8597227766525100338_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6819523382092000914_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6819523382092000914_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6819523382092000914_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8089170322185316264_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:51 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8089170322185316264_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8089170322185316264_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4724486319889730019_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4724486319889730019_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4724486319889730019_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6598509784156411285_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6598509784156411285_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6598509784156411285_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5550158876877902068_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5550158876877902068_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5550158876877902068_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5396199211660400092_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5396199211660400092_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5396199211660400092_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6091863905692269610_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6091863905692269610_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6091863905692269610_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4665764494341863421_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4665764494341863421_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4665764494341863421_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6365825502594155993_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6365825502594155993_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6365825502594155993_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6223326672393386224_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6223326672393386224_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6223326672393386224_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6983751111036327753_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6983751111036327753_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6983751111036327753_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4892298319675675972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:52 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4892298319675675972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4892298319675675972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6529758024297897433_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6529758024297897433_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6529758024297897433_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7315737574243722208_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7315737574243722208_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7315737574243722208_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5917854881101514259_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5917854881101514259_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5917854881101514259_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8920198964783835286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8920198964783835286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8920198964783835286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7758031868923649253_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7758031868923649253_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7758031868923649253_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8934721012691762334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8934721012691762334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8934721012691762334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8365060438058704861_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8365060438058704861_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8365060438058704861_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7781018796752692384_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7781018796752692384_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7781018796752692384_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5435170945955153340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5435170945955153340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5435170945955153340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8692632578386776018_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:53 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8692632578386776018_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8692632578386776018_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4986408032567985835_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_4986408032567985835_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4986408032567985835_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5140498422845221961_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5140498422845221961_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5140498422845221961_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5364738583410161244_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5364738583410161244_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5364738583410161244_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4941082175677742960_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4941082175677742960_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4941082175677742960_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6067402874622148044_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6067402874622148044_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6067402874622148044_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7820560952647059989_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7820560952647059989_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7820560952647059989_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7848222975155651684_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7848222975155651684_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7848222975155651684_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6175727663833231157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6175727663833231157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6175727663833231157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8248804441069989542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8248804441069989542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8248804441069989542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5294849977180421508_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:54 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5294849977180421508_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5294849977180421508_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6513132995668414717_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6513132995668414717_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6513132995668414717_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7475185677958136974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7475185677958136974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7475185677958136974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7192722256845746829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7192722256845746829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7192722256845746829_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8834311031980069996_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8834311031980069996_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8834311031980069996_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5318351942112337417_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5318351942112337417_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5318351942112337417_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7803174387739088549_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7803174387739088549_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7803174387739088549_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8829581017557821535_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8829581017557821535_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8829581017557821535_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6874856365773017653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6874856365773017653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6874856365773017653_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6507102522163307817_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6507102522163307817_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6507102522163307817_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7483677619000579464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:55 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7483677619000579464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7483677619000579464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7803909090341623971_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7803909090341623971_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7803909090341623971_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4739764070716356056_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4739764070716356056_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4739764070716356056_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5118468900883102352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5118468900883102352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5118468900883102352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8338760000121738200_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8338760000121738200_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8338760000121738200_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8561180472461966186_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8561180472461966186_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8561180472461966186_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5320163811816566419_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5320163811816566419_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5320163811816566419_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6731987794511815256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6731987794511815256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6731987794511815256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7073420412740024764_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7073420412740024764_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7073420412740024764_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5837839952949243340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5837839952949243340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5837839952949243340_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8281909090887290336_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:56 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8281909090887290336_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8281909090887290336_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6055524466093182749_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6055524466093182749_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6055524466093182749_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7044885610208964617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7044885610208964617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7044885610208964617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4982046573581987311_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4982046573581987311_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4982046573581987311_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8970025606581765287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8970025606581765287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8970025606581765287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7538192871344925351_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7538192871344925351_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7538192871344925351_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5221626106846635651_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5221626106846635651_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5221626106846635651_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6955831992704607547_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6955831992704607547_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6955831992704607547_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9054979757242321403_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9054979757242321403_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9054979757242321403_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5987371551834493413_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5987371551834493413_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5987371551834493413_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6296242560028910857_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:57 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6296242560028910857_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6296242560028910857_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8763693709206432541_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8763693709206432541_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8763693709206432541_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4755011365987101089_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4755011365987101089_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4755011365987101089_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8485792795647897872_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8485792795647897872_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8485792795647897872_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6851587930014992287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6851587930014992287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6851587930014992287_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5845724338321658448_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5845724338321658448_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5845724338321658448_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6131524625084222295_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6131524625084222295_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6131524625084222295_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8786072791778896378_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8786072791778896378_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8786072791778896378_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8016391051539651083_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8016391051539651083_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8016391051539651083_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4684934629940132907_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4684934629940132907_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4684934629940132907_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:58 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5290127804753807031_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5290127804753807031_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5290127804753807031_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6149084804717771619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_6149084804717771619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6149084804717771619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4716807140630825713_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4716807140630825713_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4716807140630825713_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7005974278305139620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7005974278305139620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7005974278305139620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6557607330773283662_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6557607330773283662_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6557607330773283662_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8864580984254451510_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8864580984254451510_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8864580984254451510_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8974781747699564542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8974781747699564542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8974781747699564542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8657663123059238172_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8657663123059238172_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8657663123059238172_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7333381701590919059_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7333381701590919059_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:47:59 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7333381701590919059_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6307716620358555617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6307716620358555617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6307716620358555617_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8082409778699899003_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8082409778699899003_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8082409778699899003_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7724177000927965634_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7724177000927965634_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7724177000927965634_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5843043476275064427_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5843043476275064427_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5843043476275064427_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8178643530868698729_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8178643530868698729_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8178643530868698729_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5086503965574807688_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5086503965574807688_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5086503965574807688_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8525327803914993736_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8525327803914993736_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8525327803914993736_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8307737791458093001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8307737791458093001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8307737791458093001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6930757276761984246_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6930757276761984246_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6930757276761984246_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5094567084779030679_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:00 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5094567084779030679_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5094567084779030679_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5029761691781187851_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5029761691781187851_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5029761691781187851_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7585919785023502286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7585919785023502286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7585919785023502286_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5671322599400087191_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5671322599400087191_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5671322599400087191_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8978626548777983256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8978626548777983256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8978626548777983256_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4621162518742808302_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_4621162518742808302_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4621162518742808302_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7991721278941348728_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7991721278941348728_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7991721278941348728_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7056601053452641517_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7056601053452641517_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7056601053452641517_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8541028731319044533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_8541028731319044533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8541028731319044533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7001929111919888658_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7001929111919888658_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7001929111919888658_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:01 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5364489244237043886_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5364489244237043886_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5364489244237043886_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7818261525406700741_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7818261525406700741_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7818261525406700741_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6293672031710017972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_6293672031710017972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6293672031710017972_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7805220136208473644_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7805220136208473644_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7805220136208473644_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5762964012541202543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5762964012541202543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5762964012541202543_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4865096352454304568_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_4865096352454304568_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4865096352454304568_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5473302556076294918_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5473302556076294918_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5473302556076294918_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7775113001107150440_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:02 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7775113001107150440_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7775113001107150440_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9194591517020938876_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_9194591517020938876_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9194591517020938876_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7364881515899590692_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7364881515899590692_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7364881515899590692_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8365464715287212113_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_8365464715287212113_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8365464715287212113_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7192658865233119659_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7192658865233119659_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7192658865233119659_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5257916837760221533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5257916837760221533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5257916837760221533_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7248379352729012873_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7248379352729012873_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7248379352729012873_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5329636247703065348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_5329636247703065348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5329636247703065348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6951152200524460879_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_6951152200524460879_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6951152200524460879_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7331155182877239229_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7331155182877239229_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7331155182877239229_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6707606930631956226_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_6707606930631956226_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6707606930631956226_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7322335517499999702_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7322335517499999702_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7322335517499999702_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6227752616268694437_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:04 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6227752616268694437_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6227752616268694437_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4920391758964885512_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4920391758964885512_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4920391758964885512_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7667426752759307999_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7667426752759307999_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7667426752759307999_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5353824710495661974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5353824710495661974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5353824710495661974_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6206367204373448348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6206367204373448348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6206367204373448348_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4800722086205379313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4800722086205379313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4800722086205379313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6539460671173023545_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6539460671173023545_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6539460671173023545_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8086992148390542750_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8086992148390542750_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8086992148390542750_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7012010390147932161_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7012010390147932161_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7012010390147932161_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6763606036224043268_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6763606036224043268_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6763606036224043268_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4733861873219261896_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4733861873219261896_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4733861873219261896_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4820045821320974887_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4820045821320974887_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4820045821320974887_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8238372825814495181_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8238372825814495181_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8238372825814495181_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7647854535537665616_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7647854535537665616_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7647854535537665616_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5785503758781616710_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5785503758781616710_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5785503758781616710_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5032077763963568001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5032077763963568001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5032077763963568001_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6672941172505586915_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6672941172505586915_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6672941172505586915_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6175459294950622987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6175459294950622987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6175459294950622987_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8602170336751192117_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8602170336751192117_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8602170336751192117_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5488229972651098213_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5488229972651098213_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5488229972651098213_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7067935067494012546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:06 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7067935067494012546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7067935067494012546_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9073541503164699726_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9073541503164699726_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9073541503164699726_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6626772913330213297_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6626772913330213297_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6626772913330213297_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6214011316319476874_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6214011316319476874_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6214011316319476874_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6868081995451640339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6868081995451640339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6868081995451640339_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7962148306142798460_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7962148306142798460_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7962148306142798460_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7703467552152896565_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7703467552152896565_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7703467552152896565_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8275087788936675603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8275087788936675603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8275087788936675603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8354171149275152556_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8354171149275152556_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8354171149275152556_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7563897635589321190_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7563897635589321190_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7563897635589321190_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8090283050957319954_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:07 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8090283050957319954_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8090283050957319954_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5096655703729754201_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5096655703729754201_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5096655703729754201_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7517710806682998002_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7285677779516333513_7517710806682998002_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7517710806682998002_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8456691737770790313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8456691737770790313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8456691737770790313_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7346497372746283157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7346497372746283157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7346497372746283157_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4866550562556775555_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4866550562556775555_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4866550562556775555_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7926715887793746228_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7926715887793746228_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7926715887793746228_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8802439434577089542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8802439434577089542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8802439434577089542_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6712094248012770382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6712094248012770382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6712094248012770382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8040998348953409184_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8040998348953409184_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8040998348953409184_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5327303314211643143_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:08 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5327303314211643143_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5327303314211643143_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5247582253194131624_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5247582253194131624_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5247582253194131624_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5851482141371279337_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5851482141371279337_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5851482141371279337_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8739096123507466369_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8739096123507466369_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8739096123507466369_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6296126235825613210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6296126235825613210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6296126235825613210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5442741993819692279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5442741993819692279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5442741993819692279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7340393682843955343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7340393682843955343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7340393682843955343_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4688713773617112885_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4688713773617112885_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4688713773617112885_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4993167392044281665_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4993167392044281665_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4993167392044281665_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5980779221515954382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5980779221515954382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5980779221515954382_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5582315613528398291_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:09 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5582315613528398291_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5582315613528398291_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6401327288999331144_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6401327288999331144_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6401327288999331144_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8576440449732120179_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8576440449732120179_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8576440449732120179_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7315476651860677037_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7315476651860677037_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7315476651860677037_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5020982663665616030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5020982663665616030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5020982663665616030_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7990029447605271232_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7990029447605271232_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7990029447605271232_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5909573562196235936_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5909573562196235936_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5909573562196235936_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7328317234562613458_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7328317234562613458_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7328317234562613458_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4939959432928198069_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4939959432928198069_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4939959432928198069_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8409038084308681367_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:10 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8409038084308681367_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8409038084308681367_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6591287694090917927_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6591287694090917927_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6591287694090917927_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6149845548094795880_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6149845548094795880_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6149845548094795880_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7810326047564586218_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7810326047564586218_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7810326047564586218_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5148685534130620241_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5148685534130620241_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5148685534130620241_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6489827173832299516_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6489827173832299516_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6489827173832299516_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8002062178230403979_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8002062178230403979_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8002062178230403979_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7945313223005538514_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7945313223005538514_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7945313223005538514_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8653810007462249459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8653810007462249459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8653810007462249459_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6596539208509267536_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6596539208509267536_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6596539208509267536_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8744230758713975838_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:11 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8744230758713975838_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8744230758713975838_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7602138950317592799_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7602138950317592799_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7602138950317592799_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7535709040602707334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:12 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7535709040602707334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:13 INFO DatabricksFileSystemV2Factory: Creating abfss file system for abfss://raw@zedlsbi01t.dfs.core.windows.net 23/06/28 07:48:13 INFO AzureBlobFileSystem:V3: Initializing AzureBlobFileSystem for abfss://raw@zedlsbi01t.dfs.core.windows.net/ with credential = ClientCredsTokenProvider;-298583247;574728210 with jvmId = 2036 23/06/28 07:48:14 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7535709040602707334_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:14 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8589093520484597682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:14 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8589093520484597682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:14 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8589093520484597682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:14 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5111208088729149603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:14 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5111208088729149603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:15 INFO InMemoryFileIndex: Start listing leaf files and directories. Size of Paths: 1; threshold: 32 23/06/28 07:48:15 INFO InMemoryFileIndex: Start listing leaf files and directories. Size of Paths: 0; threshold: 32 23/06/28 07:48:15 INFO InMemoryFileIndex: It took 61 ms to list leaf files for 1 paths. 23/06/28 07:48:15 INFO InMemoryFileIndex: Start listing leaf files and directories. Size of Paths: 1; threshold: 32 23/06/28 07:48:15 INFO InMemoryFileIndex: Start listing leaf files and directories. Size of Paths: 0; threshold: 32 23/06/28 07:48:15 INFO InMemoryFileIndex: It took 18 ms to list leaf files for 1 paths. 23/06/28 07:48:15 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:48:15 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 519.8 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 191.3 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 18.1 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.60.24.15:40507 (size: 18.1 KiB, free: 750.7 GiB) 23/06/28 07:48:16 INFO SparkContext: Created broadcast 1 from writeExternal at ObjectOutputStream.java:1459 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 14.3 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.60.24.15:40507 (size: 14.3 KiB, free: 750.7 GiB) 23/06/28 07:48:16 INFO SparkContext: Created broadcast 0 from load at NativeMethodAccessorImpl.java:0 23/06/28 07:48:16 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 4194304 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:48:16 INFO SparkContext: Starting job: load at NativeMethodAccessorImpl.java:0 23/06/28 07:48:16 INFO DAGScheduler: Got job 0 (load at NativeMethodAccessorImpl.java:0) with 1 output partitions 23/06/28 07:48:16 INFO DAGScheduler: Final stage: ResultStage 0 (load at NativeMethodAccessorImpl.java:0) 23/06/28 07:48:16 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:48:16 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:16 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at load at NativeMethodAccessorImpl.java:0), which has no missing parents 23/06/28 07:48:16 INFO RddExecutionContext: Config field is not HadoopMapRedWriteConfigUtil or HadoopMapReduceWriteConfigUtil, it's org.apache.spark.sql.catalyst.json.JsonInferSchema$$Lambda$7139/248019428 23/06/28 07:48:16 INFO RddExecutionContext: Found job conf from RDD Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-rbf-default.xml, hdfs-site.xml, hdfs-rbf-site.xml 23/06/28 07:48:16 INFO RddExecutionContext: Found output path null from RDD MapPartitionsRDD[3] at load at NativeMethodAccessorImpl.java:0 23/06/28 07:48:16 INFO RddExecutionContext: RDDs are empty: skipping sending OpenLineage event 23/06/28 07:48:16 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:16 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:16 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:16 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at load at NativeMethodAccessorImpl.java:0) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:16 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks resource profile 0 23/06/28 07:48:16 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 7670323585580043917, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 7670323585580043917. Created 7670323585580043917 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:48:16 INFO FairSchedulableBuilder: Added task set TaskSet_0.0 tasks to pool 7670323585580043917 23/06/28 07:48:16 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 165.6 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 37.8 KiB, free 750.7 GiB) 23/06/28 07:48:16 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.60.24.15:40507 (size: 37.8 KiB, free: 750.7 GiB) 23/06/28 07:48:16 INFO SparkContext: Created broadcast 2 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:16 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.60.24.13:44035 (size: 37.8 KiB, free: 487.7 GiB) 23/06/28 07:48:18 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.60.24.13:44035 (size: 14.3 KiB, free: 487.7 GiB) 23/06/28 07:48:18 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.60.24.13:44035 (size: 18.1 KiB, free: 487.7 GiB) 23/06/28 07:48:23 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 6798 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:23 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:23 INFO DAGScheduler: ResultStage 0 (load at NativeMethodAccessorImpl.java:0) finished in 6.932 s 23/06/28 07:48:23 INFO DAGScheduler: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:23 INFO TaskSchedulerImpl: Killing all running tasks in stage 0: Stage finished 23/06/28 07:48:23 INFO RddExecutionContext: RDDs are empty: skipping sending OpenLineage event 23/06/28 07:48:23 INFO DAGScheduler: Job 0 finished: load at NativeMethodAccessorImpl.java:0, took 7.011209 s 23/06/28 07:48:23 INFO ClusterLoadMonitor: Added query with execution ID:68. Current active queries:1 23/06/28 07:48:23 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:48:23 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:48:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:24 INFO CodeGenerator: Code generated in 322.159988 ms 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 519.0 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 14.3 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.60.24.15:40507 (size: 14.3 KiB, free: 750.7 GiB) 23/06/28 07:48:24 INFO SparkContext: Created broadcast 3 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:48:24 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 4194304 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:48:24 INFO DAGScheduler: Registering RDD 7 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 0 23/06/28 07:48:24 INFO DAGScheduler: Got map stage job 1 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:24 INFO DAGScheduler: Final stage: ShuffleMapStage 1 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:24 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:48:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:24 INFO DAGScheduler: Submitting ShuffleMapStage 1 (MapPartitionsRDD[7] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:24 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 1 (MapPartitionsRDD[7] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:24 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks resource profile 0 23/06/28 07:48:24 INFO FairSchedulableBuilder: Added task set TaskSet_1.0 tasks to pool 7670323585580043917 23/06/28 07:48:24 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 142.9 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 27.0 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.60.24.15:40507 (size: 27.0 KiB, free: 750.7 GiB) 23/06/28 07:48:24 INFO SparkContext: Created broadcast 4 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.60.24.13:44035 (size: 27.0 KiB, free: 487.7 GiB) 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.60.24.13:44035 (size: 14.3 KiB, free: 487.7 GiB) 23/06/28 07:48:24 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 275 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:24 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:24 INFO DAGScheduler: ShuffleMapStage 1 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.298 s 23/06/28 07:48:24 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:24 INFO DAGScheduler: running: Set() 23/06/28 07:48:24 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:24 INFO DAGScheduler: failed: Set() 23/06/28 07:48:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:24 INFO CodeGenerator: Code generated in 15.179427 ms 23/06/28 07:48:24 INFO SparkContext: Starting job: count at NativeMethodAccessorImpl.java:0 23/06/28 07:48:24 INFO DAGScheduler: Got job 2 (count at NativeMethodAccessorImpl.java:0) with 1 output partitions 23/06/28 07:48:24 INFO DAGScheduler: Final stage: ResultStage 3 (count at NativeMethodAccessorImpl.java:0) 23/06/28 07:48:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 2) 23/06/28 07:48:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:24 INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[9] at count at NativeMethodAccessorImpl.java:0), which has no missing parents 23/06/28 07:48:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:24 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (MapPartitionsRDD[9] at count at NativeMethodAccessorImpl.java:0) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:24 INFO TaskSchedulerImpl: Adding task set 3.0 with 1 tasks resource profile 0 23/06/28 07:48:24 INFO FairSchedulableBuilder: Added task set TaskSet_3.0 tasks to pool 7670323585580043917 23/06/28 07:48:24 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 2) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 129.8 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 21.4 KiB, free 750.7 GiB) 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on 10.60.24.15:40507 (size: 21.4 KiB, free: 750.7 GiB) 23/06/28 07:48:24 INFO SparkContext: Created broadcast 5 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on 10.60.24.10:41041 (size: 21.4 KiB, free: 487.7 GiB) 23/06/28 07:48:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.60.24.10:44288 23/06/28 07:48:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:48:26 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 2) in 1751 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:26 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:26 INFO DAGScheduler: ResultStage 3 (count at NativeMethodAccessorImpl.java:0) finished in 1.758 s 23/06/28 07:48:26 INFO DAGScheduler: Job 2 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:26 INFO TaskSchedulerImpl: Killing all running tasks in stage 3: Stage finished 23/06/28 07:48:26 INFO DAGScheduler: Job 2 finished: count at NativeMethodAccessorImpl.java:0, took 1.765644 s 23/06/28 07:48:26 INFO ClusterLoadMonitor: Removed query with execution ID:68. Current active queries:0 23/06/28 07:48:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5111208088729149603_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8581440613166478682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8581440613166478682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8581440613166478682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6839034682864527112_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6839034682864527112_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6839034682864527112_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7087089136427375609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7087089136427375609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7087089136427375609_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8251156071774454335_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:26 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8251156071774454335_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8251156071774454335_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9017128231788457222_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9017128231788457222_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9017128231788457222_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6794394286636485149_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6794394286636485149_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6794394286636485149_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5646660277472725125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:27 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5646660277472725125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5646660277472725125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8616930761687503352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8616930761687503352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8616930761687503352_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5900399764020723464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5900399764020723464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5900399764020723464_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4990533830421282855_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4990533830421282855_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4990533830421282855_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6961260005824758539_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6961260005824758539_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6961260005824758539_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6188843123526902079_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6188843123526902079_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6188843123526902079_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7101124837899235676_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7101124837899235676_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7101124837899235676_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5719102187122897125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5719102187122897125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5719102187122897125_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5639201282062642231_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5639201282062642231_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5639201282062642231_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6546001544594119375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6546001544594119375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6546001544594119375_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6354960226192657393_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:28 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6354960226192657393_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6354960226192657393_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8542875767756928776_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8542875767756928776_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8542875767756928776_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8594298446681072995_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8594298446681072995_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8594298446681072995_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5823222871049593654_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5823222871049593654_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5823222871049593654_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5731131620764642771_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5731131620764642771_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5731131620764642771_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6751777486755173461_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6751777486755173461_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6751777486755173461_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7359152203207608341_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7359152203207608341_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7359152203207608341_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8211305869258274787_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8211305869258274787_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8211305869258274787_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7126278011217576139_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7126278011217576139_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7126278011217576139_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8572175094018963279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8572175094018963279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8572175094018963279_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5252887265462605695_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:29 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5252887265462605695_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5252887265462605695_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8928625748723229619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8928625748723229619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8928625748723229619_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8662087892258851142_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8662087892258851142_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8662087892258851142_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8767863850860855841_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8767863850860855841_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8767863850860855841_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6348969222042191986_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6348969222042191986_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6348969222042191986_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5696006624686633784_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5696006624686633784_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5696006624686633784_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5994731477428338586_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5994731477428338586_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5994731477428338586_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6823839049194331901_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6823839049194331901_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6823839049194331901_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5403719721529618223_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5403719721529618223_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5403719721529618223_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5194636383764067319_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5194636383764067319_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5194636383764067319_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5322803019526907227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:30 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5322803019526907227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5322803019526907227_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7911477498208442821_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7911477498208442821_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7911477498208442821_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7830701958189572528_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7830701958189572528_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7830701958189572528_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6468392145912107210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6468392145912107210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6468392145912107210_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7978801709417102308_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7978801709417102308_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7978801709417102308_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5070369106417553072_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5070369106417553072_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5070369106417553072_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6208578673837438947_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6208578673837438947_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6208578673837438947_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7953161496576337307_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7953161496576337307_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7953161496576337307_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5547264828676810500_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5547264828676810500_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5547264828676810500_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8605303971633535115_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8605303971633535115_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8605303971633535115_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:31 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9068147280383486682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9068147280383486682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9068147280383486682_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6231326048166744583_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6231326048166744583_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6231326048166744583_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4933343835893786104_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4933343835893786104_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4933343835893786104_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4690261122778901012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4690261122778901012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4690261122778901012_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8121812690110823849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8121812690110823849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8121812690110823849_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6722854749180517290_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6722854749180517290_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6722854749180517290_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7543015865088511734_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7543015865088511734_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7543015865088511734_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_6388270009953954391_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:32 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_6388270009953954391_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_6388270009953954391_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_4660812125659554731_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_4660812125659554731_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_4660812125659554731_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_7498572848159748281_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_7498572848159748281_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_7498572848159748281_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_8469927463799253158_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_8469927463799253158_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:48:35 INFO ClusterLoadMonitor: Added query with execution ID:69. Current active queries:1 23/06/28 07:48:35 INFO HiveMetaStore: 1: get_database: raw 23/06/28 07:48:35 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:48:35 INFO HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:48:35 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:48:35 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:48:35 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:48:35 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:48:35.5Z","run":{"runId":"61ef6378-dc0c-411d-8e02-bfd1bbb2e7e6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.CreateNamespace","num-children":1,"name":0,"ifNotExists":true,"properties":null},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedNamespace","num-children":0,"catalog":null,"namespace":"[raw]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_create_namespace","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:48:35 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:48:35 INFO ClusterLoadMonitor: Removed query with execution ID:69. Current active queries:0 23/06/28 07:48:35 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:48:35 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:48:35.559Z","run":{"runId":"61ef6378-dc0c-411d-8e02-bfd1bbb2e7e6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.CreateNamespace","num-children":1,"name":0,"ifNotExists":true,"properties":null},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedNamespace","num-children":0,"catalog":null,"namespace":"[raw]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_create_namespace","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:48:35 INFO HiveMetaStore: 1: get_database: global_temp 23/06/28 07:48:35 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: global_temp 23/06/28 07:48:35 ERROR RetryingHMSHandler: NoSuchObjectException(message:There is no database named global_temp) at org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:508) at org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:519) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108) at com.sun.proxy.$Proxy73.getDatabase(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStore.java:796) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105) at com.sun.proxy.$Proxy75.get_database(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:949) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy76.getDatabase(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1165) at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1154) at org.apache.spark.sql.hive.client.Shim_v0_12.databaseExists(HiveShim.scala:619) at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$databaseExists$1(HiveClientImpl.scala:450) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:347) at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$retryLocked$1(HiveClientImpl.scala:247) at org.apache.spark.sql.hive.client.HiveClientImpl.synchronizeOnObject(HiveClientImpl.scala:284) at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:239) at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:327) at org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala:450) at org.apache.spark.sql.hive.client.PoolingHiveClient.$anonfun$databaseExists$1(PoolingHiveClient.scala:321) at org.apache.spark.sql.hive.client.PoolingHiveClient.$anonfun$databaseExists$1$adapted(PoolingHiveClient.scala:320) at org.apache.spark.sql.hive.client.PoolingHiveClient.withHiveClient(PoolingHiveClient.scala:149) at org.apache.spark.sql.hive.client.PoolingHiveClient.databaseExists(PoolingHiveClient.scala:320) at org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$databaseExists$1(HiveExternalCatalog.scala:313) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$withClient$2(HiveExternalCatalog.scala:154) at org.apache.spark.sql.hive.HiveExternalCatalog.maybeSynchronized(HiveExternalCatalog.scala:115) at org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$withClient$1(HiveExternalCatalog.scala:153) at com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressReporter.scala:377) at com.databricks.backend.daemon.driver.ProgressReporter$.withStatusCode(ProgressReporter.scala:363) at com.databricks.spark.util.SparkDatabricksProgressReporter$.withStatusCode(ProgressReporter.scala:34) at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:152) at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:313) at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.$anonfun$databaseExists$1(ExternalCatalogWithListener.scala:84) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(ExternalCatalogWithListener.scala:84) at org.apache.spark.sql.internal.SharedState.$anonfun$globalTempViewExternalCatalogNameCheck$1(SharedState.scala:354) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at scala.util.Try$.apply(Try.scala:213) at org.apache.spark.sql.internal.SharedState.globalTempViewExternalCatalogNameCheck(SharedState.scala:354) at org.apache.spark.sql.internal.SharedState.globalTempViewManager$lzycompute(SharedState.scala:382) at org.apache.spark.sql.internal.SharedState.globalTempViewManager(SharedState.scala:378) at org.apache.spark.sql.hive.HiveSessionStateBuilder.$anonfun$hiveCatalog$2(HiveSessionStateBuilder.scala:76) at org.apache.spark.sql.catalyst.catalog.SessionCatalogImpl.globalTempViewManager$lzycompute(SessionCatalog.scala:550) at org.apache.spark.sql.catalyst.catalog.SessionCatalogImpl.globalTempViewManager(SessionCatalog.scala:550) at org.apache.spark.sql.catalyst.catalog.SessionCatalogImpl.isGlobalTempViewDB(SessionCatalog.scala:1559) at org.apache.spark.sql.catalyst.catalog.SessionCatalogImpl.getRawLocalOrGlobalTempView(SessionCatalog.scala:1254) at org.apache.spark.sql.catalyst.catalog.DelegatingSessionCatalog.getRawLocalOrGlobalTempView(DelegatingSessionCatalog.scala:240) at org.apache.spark.sql.catalyst.catalog.DelegatingSessionCatalog.getRawLocalOrGlobalTempView$(DelegatingSessionCatalog.scala:239) at com.databricks.sql.managedcatalog.ManagedCatalogSessionCatalog.getRawLocalOrGlobalTempView(ManagedCatalogSessionCatalog.scala:69) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.lookupTempView(Analyzer.scala:1577) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveRelations$$lookupTableOrView(Analyzer.scala:1609) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply0$1.applyOrElse(Analyzer.scala:1562) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply0$1.applyOrElse(Analyzer.scala:1477) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$3(AnalysisHelper.scala:139) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:106) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$1(AnalysisHelper.scala:139) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:354) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning(AnalysisHelper.scala:135) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning$(AnalysisHelper.scala:131) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsUpWithPruning(LogicalPlan.scala:31) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$2(AnalysisHelper.scala:136) at org.apache.spark.sql.catalyst.trees.UnaryLike.mapChildren(TreeNode.scala:1182) at org.apache.spark.sql.catalyst.trees.UnaryLike.mapChildren$(TreeNode.scala:1181) at org.apache.spark.sql.catalyst.plans.logical.DescribeRelation.mapChildren(v2Commands.scala:526) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$1(AnalysisHelper.scala:136) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:354) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning(AnalysisHelper.scala:135) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning$(AnalysisHelper.scala:131) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsUpWithPruning(LogicalPlan.scala:31) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply0(Analyzer.scala:1477) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:1453) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:1395) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$4(RuleExecutor.scala:218) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$3(RuleExecutor.scala:218) at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126) at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122) at scala.collection.immutable.List.foldLeft(List.scala:91) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$2(RuleExecutor.scala:215) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeBatch$1(RuleExecutor.scala:207) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$8(RuleExecutor.scala:277) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$8$adapted(RuleExecutor.scala:277) at scala.collection.immutable.List.foreach(List.scala:431) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:277) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:194) at org.apache.spark.sql.catalyst.analysis.Analyzer.executeSameContext(Analyzer.scala:360) at org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$execute$1(Analyzer.scala:353) at org.apache.spark.sql.catalyst.analysis.AnalysisContext$.withNewAnalysisContext(Analyzer.scala:260) at org.apache.spark.sql.catalyst.analysis.Analyzer.execute(Analyzer.scala:353) at org.apache.spark.sql.catalyst.analysis.Analyzer.execute(Analyzer.scala:281) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$executeAndTrack$1(RuleExecutor.scala:186) at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:154) at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeAndTrack(RuleExecutor.scala:186) at org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$executeAndCheck$1(Analyzer.scala:333) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:361) at org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:332) at org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:153) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:340) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$3(QueryExecution.scala:368) at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:803) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:368) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:365) at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:147) at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:147) at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:137) at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:109) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.SparkSession.$anonfun$withActiveAndFrameProfiler$1(SparkSession.scala:1045) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80) at org.apache.spark.sql.SparkSession.withActiveAndFrameProfiler(SparkSession.scala:1045) at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:107) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:841) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1038) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:830) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:864) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380) at py4j.Gateway.invoke(Gateway.java:306) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:195) at py4j.ClientServerConnection.run(ClientServerConnection.java:115) at java.lang.Thread.run(Thread.java:750) 23/06/28 07:48:35 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:35 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:36 INFO DeltaLog: Loading version 84 starting from checkpoint version 80. 23/06/28 07:48:36 INFO SnapshotEdge: [tableId=c6a0ba0b-caf9-49b7-b9df-add8630aa320] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=84, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,84,WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687937151000), checksumOpt=Some(VersionChecksum(Some(9b37a24a-a77a-4a22-89c5-05aa9fd62db3),75338,1,None,None,1,1,Some(List()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@35ef559b,[J@57df4e0e)),None,Some(List(AddFile(year=2023/month=06/day=28/part-00000-d26d832b-f936-48d3-a0bb-bfcbed64446c.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75338,1687937150000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687937150000000, MIN_INSERTION_TIME -> 1687937150000000, MAX_INSERTION_TIME -> 1687937150000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:48:36 INFO ClusterLoadMonitor: Added query with execution ID:70. Current active queries:1 23/06/28 07:48:36 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:37 INFO CodeGenerator: Code generated in 45.163244 ms 23/06/28 07:48:37 INFO ClusterLoadMonitor: Removed query with execution ID:70. Current active queries:0 23/06/28 07:48:37 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:37 INFO HiveMetaStore: 1: get_database: raw 23/06/28 07:48:37 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:48:37 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO ClusterLoadMonitor: Added query with execution ID:71. Current active queries:1 23/06/28 07:48:37 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:48:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:48:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:48:37 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:48:37.178Z","run":{"runId":"a30efe08-5875-4c12-b7ed-8539e506e38f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.directory.VacuumTableCommand","num-children":0,"table":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw"},"dryRun":false,"options":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_vacuum_table_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:48:37 INFO VacuumCommand: Starting garbage collection (dryRun = false) of untracked files older than 21 Jun 2023 07:48:37 GMT in dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta 23/06/28 07:48:37 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 508.5 KiB, free 750.7 GiB) 23/06/28 07:48:37 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 13.6 KiB, free 750.7 GiB) 23/06/28 07:48:37 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 10.60.24.15:40507 (size: 13.6 KiB, free: 750.7 GiB) 23/06/28 07:48:37 INFO SparkContext: Created broadcast 6 from broadcast at VacuumCommand.scala:130 23/06/28 07:48:37 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 508.5 KiB, free 750.7 GiB) 23/06/28 07:48:37 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 13.6 KiB, free 750.7 GiB) 23/06/28 07:48:37 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on 10.60.24.15:40507 (size: 13.6 KiB, free: 750.7 GiB) 23/06/28 07:48:37 INFO SparkContext: Created broadcast 7 from broadcast at DeltaLog.scala:634 23/06/28 07:48:37 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 27842) 23/06/28 07:48:37 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 4, totalFileSize: 5749) 23/06/28 07:48:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:48:38 INFO CodeGenerator: Code generated in 17.508565 ms 23/06/28 07:48:38 INFO CodeGenerator: Code generated in 18.419038 ms 23/06/28 07:48:38 INFO CodeGenerator: Code generated in 147.626421 ms 23/06/28 07:48:38 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 31.8 KiB, free 750.7 GiB) 23/06/28 07:48:38 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 14.1 KiB, free 750.7 GiB) 23/06/28 07:48:38 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on 10.60.24.15:40507 (size: 14.1 KiB, free: 750.7 GiB) 23/06/28 07:48:38 INFO SparkContext: Created broadcast 8 from withNewExecutionId at StateCache.scala:61 23/06/28 07:48:38 INFO CodeGenerator: Code generated in 66.799883 ms 23/06/28 07:48:38 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 39.0 KiB, free 750.7 GiB) 23/06/28 07:48:38 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 12.2 KiB, free 750.7 GiB) 23/06/28 07:48:38 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on 10.60.24.15:40507 (size: 12.2 KiB, free: 750.7 GiB) 23/06/28 07:48:38 INFO SparkContext: Created broadcast 9 from withNewExecutionId at StateCache.scala:61 23/06/28 07:48:38 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:48:38 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:48:38 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'. 23/06/28 07:48:38 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:38 INFO NativeLibraryLoader: Loaded library libparquet-cpp.so from /local_disk0/tmp/load-from-jar3054595787528898143libparquet-cpp.so with glog sink /databricks/driver/logs 23/06/28 07:48:39 INFO CodeGenerator: Code generated in 68.190625 ms 23/06/28 07:48:39 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 607.6 KiB, free 750.7 GiB) 23/06/28 07:48:39 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.7 GiB) 23/06/28 07:48:39 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:48:39 INFO SparkContext: Created broadcast 10 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:48:39 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:48:39 INFO RocksDBLoader: RocksDB library loading thread started 23/06/28 07:48:39 INFO RocksDBLoader: RocksDB library loading thread finished successfully 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] RocksDB version: 6.28.2 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Compile date 2022-02-04 22:23:17 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] DB SUMMARY 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] DB Session ID: BV3GH2XM6V5FRK4JPRUG 23/06/28 07:48:39 ERROR RocksDBExternalMap: [Native-3] Error when reading /local_disk0/dbio_cache_root_2036-68e624c7-c93d-4506-974e-86aa5a439c2c/locality_assignments-8f2d551b-5702-4421-909d-daaca033aa39/386118931645 dir 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] SST files in /local_disk0/dbio_cache_root_2036-68e624c7-c93d-4506-974e-86aa5a439c2c/locality_assignments-8f2d551b-5702-4421-909d-daaca033aa39/386118931645 dir, Total Num: 0, files: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Write Ahead Log file in /local_disk0/dbio_cache_root_2036-68e624c7-c93d-4506-974e-86aa5a439c2c/locality_assignments-8f2d551b-5702-4421-909d-daaca033aa39/386118931645: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.error_if_exists: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.create_if_missing: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.paranoid_checks: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.flush_verify_memtable_count: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.track_and_verify_wals_in_manifest: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.env: 0x7edc821f5780 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.fs: PosixFileSystem 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.info_log: 0x7edc6c0b4628 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_file_opening_threads: 16 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.statistics: (nil) 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.use_fsync: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_log_file_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_manifest_file_size: 1073741824 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.log_file_time_to_roll: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.keep_log_file_num: 1000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.recycle_log_file_num: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_fallocate: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_mmap_reads: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_mmap_writes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.use_direct_reads: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.use_direct_io_for_flush_and_compaction: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.create_missing_column_families: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.db_log_dir: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.wal_dir: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.table_cache_numshardbits: 6 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.WAL_ttl_seconds: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.WAL_size_limit_MB: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_write_batch_group_size_bytes: 1048576 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.manifest_preallocation_size: 4194304 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.is_fd_close_on_exec: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.advise_random_on_open: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.experimental_mempurge_threshold: 0.000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.db_write_buffer_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.write_buffer_manager: 0x7edc6c0a33d0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.access_hint_on_compaction_start: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.new_table_reader_for_compaction_inputs: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.random_access_max_buffer_size: 1048576 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.use_adaptive_mutex: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.rate_limiter: (nil) 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.sst_file_manager.rate_bytes_per_sec: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.wal_recovery_mode: 2 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.enable_thread_tracking: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.enable_pipelined_write: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.unordered_write: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_concurrent_memtable_write: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.enable_write_thread_adaptive_yield: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.write_thread_max_yield_usec: 100 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.write_thread_slow_yield_usec: 3 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.row_cache: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.wal_filter: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.avoid_flush_during_recovery: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_ingest_behind: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.preserve_deletes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.two_write_queues: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.manual_wal_flush: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.atomic_flush: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.avoid_unnecessary_blocking_io: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.persist_stats_to_disk: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.write_dbid_to_manifest: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.log_readahead_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.file_checksum_gen_factory: Unknown 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.best_efforts_recovery: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bgerror_resume_count: 2147483647 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bgerror_resume_retry_interval: 1000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.allow_data_in_errors: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.db_host_id: __hostname__ 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_background_jobs: 2 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_background_compactions: -1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_subcompactions: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.avoid_flush_during_shutdown: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.writable_file_max_buffer_size: 1048576 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.delayed_write_rate : 16777216 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_total_wal_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.delete_obsolete_files_period_micros: 21600000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.stats_dump_period_sec: 600 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.stats_persist_period_sec: 600 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.stats_history_buffer_size: 1048576 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_open_files: -1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bytes_per_sync: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.wal_bytes_per_sync: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.strict_bytes_per_sync: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_readahead_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_background_flushes: -1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Compression algorithms supported: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kZSTDNotFinalCompression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kZSTD supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kXpressCompression supported: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kLZ4HCCompression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kLZ4Compression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kBZip2Compression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kZlibCompression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] kSnappyCompression supported: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Fast CRC32 supported: Not supported on x86 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/db_impl/db_impl_open.cc:307] Creating manifest 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/version_set.cc:4847] Recovering from manifest file: /local_disk0/dbio_cache_root_2036-68e624c7-c93d-4506-974e-86aa5a439c2c/locality_assignments-8f2d551b-5702-4421-909d-daaca033aa39/386118931645/MANIFEST-000001 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/column_family.cc:607] --------------- Options for column family [default]: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.comparator: leveldb.BytewiseComparator 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.merge_operator: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_filter: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_filter_factory: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.sst_partitioner_factory: None 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.memtable_factory: SkipListFactory 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.table_factory: BlockBasedTable 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x7edc6c0b4d30) cache_index_and_filter_blocks: 0 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x7edc6c0b4700 block_cache_name: LRUCache block_cache_options: capacity : 8388608 num_shard_bits : 4 strict_capacity_limit : 0 memory_allocator : None high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: nullptr whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.write_buffer_size: 67108864 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_write_buffer_number: 2 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression: Snappy 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression: Disabled 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.prefix_extractor: nullptr 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.memtable_insert_with_hint_prefix_extractor: nullptr 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.num_levels: 7 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.min_write_buffer_number_to_merge: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_write_buffer_number_to_maintain: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_write_buffer_size_to_maintain: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.window_bits: -14 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.level: 32767 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.strategy: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.max_dict_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.zstd_max_train_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.parallel_threads: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.enabled: false 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.window_bits: -14 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.level: 32767 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.strategy: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.max_dict_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.zstd_max_train_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.parallel_threads: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.enabled: false 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compression_opts.max_dict_buffer_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.level0_file_num_compaction_trigger: 4 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.level0_slowdown_writes_trigger: 20 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.level0_stop_writes_trigger: 36 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.target_file_size_base: 67108864 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.target_file_size_multiplier: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_base: 268435456 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.level_compaction_dynamic_level_bytes: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier: 10.000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[0]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[1]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[2]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[3]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[4]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[5]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_bytes_for_level_multiplier_addtl[6]: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_sequential_skip_in_iterations: 8 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_compaction_bytes: 1677721600 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.arena_block_size: 1048576 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.soft_pending_compaction_bytes_limit: 68719476736 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.hard_pending_compaction_bytes_limit: 274877906944 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.rate_limit_delay_max_milliseconds: 100 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.disable_auto_compactions: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_style: kCompactionStyleLevel 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_pri: kMinOverlappingRatio 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.size_ratio: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.min_merge_width: 2 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.max_merge_width: 4294967295 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.max_size_amplification_percent: 200 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.compression_size_percent: -1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_fifo.max_table_files_size: 1073741824 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.compaction_options_fifo.allow_compaction: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.table_properties_collectors: 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.inplace_update_support: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.inplace_update_num_locks: 10000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.memtable_prefix_bloom_size_ratio: 0.000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.memtable_whole_key_filtering: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.memtable_huge_page_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.bloom_locality: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.max_successive_merges: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.optimize_filters_for_hits: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.paranoid_file_checks: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.force_consistency_checks: 1 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.report_bg_io_stats: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.ttl: 2592000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.periodic_compaction_seconds: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.enable_blob_files: false 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.min_blob_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.blob_file_size: 268435456 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.blob_compression_type: NoCompression 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.enable_blob_garbage_collection: false 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.blob_garbage_collection_age_cutoff: 0.250000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.blob_garbage_collection_force_threshold: 1.000000 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] Options.blob_compaction_readahead_size: 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/version_set.cc:4895] Recovered from manifest file:/local_disk0/dbio_cache_root_2036-68e624c7-c93d-4506-974e-86aa5a439c2c/locality_assignments-8f2d551b-5702-4421-909d-daaca033aa39/386118931645/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/version_set.cc:4904] Column family [default] (ID 0), log number is 0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/version_set.cc:4385] Creating manifest 4 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/db_impl/db_impl_open.cc:1793] SstFileManager instance 0x7edc6c06fd40 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] DB pointer 0x7edc6c0a38f0 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/db_impl/db_impl.cc:1005] ------- DUMPING STATS ------- 23/06/28 07:48:39 INFO RocksDBExternalMap: [Native-1] [/db_impl/db_impl.cc:1006] ** DB Stats ** Uptime(secs): 0.0 total, 0.0 interval Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache LRUCache@0x7edc6c0b4700#2036 capacity: 8.00 MB collections: 1 last_copies: 0 last_secs: 3.9e-05 secs_since: 0 Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [default] ** 23/06/28 07:48:39 INFO CodeGenerator: Code generated in 30.668495 ms 23/06/28 07:48:39 INFO DAGScheduler: Registering RDD 16 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 1 23/06/28 07:48:39 INFO DAGScheduler: Got map stage job 3 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 9 output partitions 23/06/28 07:48:39 INFO DAGScheduler: Final stage: ShuffleMapStage 4 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:39 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:48:39 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:39 INFO DAGScheduler: Submitting ShuffleMapStage 4 (MapPartitionsRDD[16] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:39 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:39 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:39 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:39 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:39 INFO DAGScheduler: Submitting 9 missing tasks from ShuffleMapStage 4 (MapPartitionsRDD[16] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8)) 23/06/28 07:48:39 INFO TaskSchedulerImpl: Adding task set 4.0 with 9 tasks resource profile 0 23/06/28 07:48:39 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 7670323585580043917, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 7670323585580043917. Created 7670323585580043917 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:48:39 INFO FairSchedulableBuilder: Added task set TaskSet_4.0 tasks to pool 7670323585580043917 23/06/28 07:48:39 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 3) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 4) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 2.0 in stage 4.0 (TID 5) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 3.0 in stage 4.0 (TID 6) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 4.0 in stage 4.0 (TID 7) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 5.0 in stage 4.0 (TID 8) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 6.0 in stage 4.0 (TID 9) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 7.0 in stage 4.0 (TID 10) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO TaskSetManager: Starting task 8.0 in stage 4.0 (TID 11) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:39 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 324.2 KiB, free 750.7 GiB) 23/06/28 07:48:39 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 79.3 KiB, free 750.7 GiB) 23/06/28 07:48:39 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on 10.60.24.15:40507 (size: 79.3 KiB, free: 750.7 GiB) 23/06/28 07:48:39 INFO SparkContext: Created broadcast 11 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:39 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on 10.60.24.10:41041 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:39 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on 10.60.24.13:44035 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:39 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.60.24.10:41041 (size: 18.1 KiB, free: 487.7 GiB) 23/06/28 07:48:39 INFO TaskSetManager: Finished task 6.0 in stage 4.0 (TID 9) in 291 ms on 10.60.24.13 (executor 1) (1/9) 23/06/28 07:48:39 INFO TaskSetManager: Finished task 2.0 in stage 4.0 (TID 5) in 299 ms on 10.60.24.13 (executor 1) (2/9) 23/06/28 07:48:39 INFO TaskSetManager: Finished task 4.0 in stage 4.0 (TID 7) in 298 ms on 10.60.24.13 (executor 1) (3/9) 23/06/28 07:48:39 INFO TaskSetManager: Finished task 8.0 in stage 4.0 (TID 11) in 300 ms on 10.60.24.13 (executor 1) (4/9) 23/06/28 07:48:40 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 4) in 641 ms on 10.60.24.10 (executor 0) (5/9) 23/06/28 07:48:40 INFO TaskSetManager: Finished task 5.0 in stage 4.0 (TID 8) in 641 ms on 10.60.24.10 (executor 0) (6/9) 23/06/28 07:48:40 INFO TaskSetManager: Finished task 7.0 in stage 4.0 (TID 10) in 641 ms on 10.60.24.10 (executor 0) (7/9) 23/06/28 07:48:40 INFO TaskSetManager: Finished task 3.0 in stage 4.0 (TID 6) in 641 ms on 10.60.24.10 (executor 0) (8/9) 23/06/28 07:48:40 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:48:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:46 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 3) in 7024 ms on 10.60.24.10 (executor 0) (9/9) 23/06/28 07:48:46 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:46 INFO DAGScheduler: ShuffleMapStage 4 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 7.114 s 23/06/28 07:48:46 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:46 INFO DAGScheduler: running: Set() 23/06/28 07:48:46 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:46 INFO DAGScheduler: failed: Set() 23/06/28 07:48:46 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:46 INFO CodeGenerator: Code generated in 55.468731 ms 23/06/28 07:48:46 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:46 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":5103,"lifetime":225108},"10.60.24.13":{"diskUsage":0,"lifetime":225378}}, ioCacheNumScanTasks={"numLocalScanTasks":1,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:48:46 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInWrite' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInWrite' instead. 23/06/28 07:48:46 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:48:46 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInRead' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInRead' instead. 23/06/28 07:48:46 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$ 23/06/28 07:48:46 INFO CodeGenerator: Code generated in 28.132996 ms 23/06/28 07:48:46 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 4.7 KiB, free 750.7 GiB) 23/06/28 07:48:46 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 2.2 KiB, free 750.7 GiB) 23/06/28 07:48:46 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on 10.60.24.15:40507 (size: 2.2 KiB, free: 750.7 GiB) 23/06/28 07:48:46 INFO SparkContext: Created broadcast 12 from cache at VacuumCommand.scala:184 23/06/28 07:48:46 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:47 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:47 INFO CodeGenerator: Code generated in 8.581415 ms 23/06/28 07:48:47 INFO DAGScheduler: Registering RDD 26 (repartition at DeltaFileOperations.scala:251) as input to shuffle 3 23/06/28 07:48:47 INFO DAGScheduler: Registering RDD 33 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 2 23/06/28 07:48:47 INFO DAGScheduler: Registering RDD 44 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 4 23/06/28 07:48:47 INFO DAGScheduler: Got map stage job 4 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 200 output partitions 23/06/28 07:48:47 INFO DAGScheduler: Final stage: ShuffleMapStage 7 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:47 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 6) 23/06/28 07:48:47 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 6) 23/06/28 07:48:47 INFO DAGScheduler: Submitting ShuffleMapStage 5 (MapPartitionsRDD[26] at repartition at DeltaFileOperations.scala:251), which has no missing parents 23/06/28 07:48:47 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:47 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:47 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:47 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:47 INFO DAGScheduler: Submitting 192 missing tasks from ShuffleMapStage 5 (MapPartitionsRDD[26] at repartition at DeltaFileOperations.scala:251) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:47 INFO TaskSchedulerImpl: Adding task set 5.0 with 192 tasks resource profile 0 23/06/28 07:48:47 INFO FairSchedulableBuilder: Added task set TaskSet_5.0 tasks to pool 7670323585580043917 23/06/28 07:48:47 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 12) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 13) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 2.0 in stage 5.0 (TID 14) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 3.0 in stage 5.0 (TID 15) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 4.0 in stage 5.0 (TID 16) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 5.0 in stage 5.0 (TID 17) (10.60.24.13, executor 1, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 6.0 in stage 5.0 (TID 18) (10.60.24.10, executor 0, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 7.0 in stage 5.0 (TID 19) (10.60.24.13, executor 1, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 8.0 in stage 5.0 (TID 20) (10.60.24.10, executor 0, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 9.0 in stage 5.0 (TID 21) (10.60.24.13, executor 1, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 10.0 in stage 5.0 (TID 22) (10.60.24.10, executor 0, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 11.0 in stage 5.0 (TID 23) (10.60.24.13, executor 1, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 12.0 in stage 5.0 (TID 24) (10.60.24.10, executor 0, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 13.0 in stage 5.0 (TID 25) (10.60.24.13, executor 1, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 14.0 in stage 5.0 (TID 26) (10.60.24.10, executor 0, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 15.0 in stage 5.0 (TID 27) (10.60.24.13, executor 1, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 16.0 in stage 5.0 (TID 28) (10.60.24.10, executor 0, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 17.0 in stage 5.0 (TID 29) (10.60.24.13, executor 1, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 18.0 in stage 5.0 (TID 30) (10.60.24.10, executor 0, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 19.0 in stage 5.0 (TID 31) (10.60.24.13, executor 1, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 20.0 in stage 5.0 (TID 32) (10.60.24.10, executor 0, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 21.0 in stage 5.0 (TID 33) (10.60.24.13, executor 1, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 22.0 in stage 5.0 (TID 34) (10.60.24.10, executor 0, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 23.0 in stage 5.0 (TID 35) (10.60.24.13, executor 1, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 24.0 in stage 5.0 (TID 36) (10.60.24.10, executor 0, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 25.0 in stage 5.0 (TID 37) (10.60.24.13, executor 1, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 26.0 in stage 5.0 (TID 38) (10.60.24.10, executor 0, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 27.0 in stage 5.0 (TID 39) (10.60.24.13, executor 1, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 28.0 in stage 5.0 (TID 40) (10.60.24.10, executor 0, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 29.0 in stage 5.0 (TID 41) (10.60.24.13, executor 1, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 30.0 in stage 5.0 (TID 42) (10.60.24.10, executor 0, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 31.0 in stage 5.0 (TID 43) (10.60.24.13, executor 1, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 32.0 in stage 5.0 (TID 44) (10.60.24.10, executor 0, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 33.0 in stage 5.0 (TID 45) (10.60.24.13, executor 1, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 34.0 in stage 5.0 (TID 46) (10.60.24.10, executor 0, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 35.0 in stage 5.0 (TID 47) (10.60.24.13, executor 1, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 36.0 in stage 5.0 (TID 48) (10.60.24.10, executor 0, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 37.0 in stage 5.0 (TID 49) (10.60.24.13, executor 1, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 38.0 in stage 5.0 (TID 50) (10.60.24.10, executor 0, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 39.0 in stage 5.0 (TID 51) (10.60.24.13, executor 1, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 40.0 in stage 5.0 (TID 52) (10.60.24.10, executor 0, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 41.0 in stage 5.0 (TID 53) (10.60.24.13, executor 1, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 42.0 in stage 5.0 (TID 54) (10.60.24.10, executor 0, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 43.0 in stage 5.0 (TID 55) (10.60.24.13, executor 1, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 44.0 in stage 5.0 (TID 56) (10.60.24.10, executor 0, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 45.0 in stage 5.0 (TID 57) (10.60.24.13, executor 1, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 46.0 in stage 5.0 (TID 58) (10.60.24.10, executor 0, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 47.0 in stage 5.0 (TID 59) (10.60.24.13, executor 1, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 48.0 in stage 5.0 (TID 60) (10.60.24.10, executor 0, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 49.0 in stage 5.0 (TID 61) (10.60.24.13, executor 1, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 50.0 in stage 5.0 (TID 62) (10.60.24.10, executor 0, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 51.0 in stage 5.0 (TID 63) (10.60.24.13, executor 1, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 52.0 in stage 5.0 (TID 64) (10.60.24.10, executor 0, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 53.0 in stage 5.0 (TID 65) (10.60.24.13, executor 1, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 54.0 in stage 5.0 (TID 66) (10.60.24.10, executor 0, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 55.0 in stage 5.0 (TID 67) (10.60.24.13, executor 1, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 56.0 in stage 5.0 (TID 68) (10.60.24.10, executor 0, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 57.0 in stage 5.0 (TID 69) (10.60.24.13, executor 1, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 58.0 in stage 5.0 (TID 70) (10.60.24.10, executor 0, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 59.0 in stage 5.0 (TID 71) (10.60.24.13, executor 1, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 60.0 in stage 5.0 (TID 72) (10.60.24.10, executor 0, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 61.0 in stage 5.0 (TID 73) (10.60.24.13, executor 1, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 62.0 in stage 5.0 (TID 74) (10.60.24.10, executor 0, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 63.0 in stage 5.0 (TID 75) (10.60.24.13, executor 1, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 64.0 in stage 5.0 (TID 76) (10.60.24.10, executor 0, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 65.0 in stage 5.0 (TID 77) (10.60.24.13, executor 1, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 66.0 in stage 5.0 (TID 78) (10.60.24.10, executor 0, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 67.0 in stage 5.0 (TID 79) (10.60.24.13, executor 1, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 68.0 in stage 5.0 (TID 80) (10.60.24.10, executor 0, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 69.0 in stage 5.0 (TID 81) (10.60.24.13, executor 1, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 70.0 in stage 5.0 (TID 82) (10.60.24.10, executor 0, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 71.0 in stage 5.0 (TID 83) (10.60.24.13, executor 1, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 72.0 in stage 5.0 (TID 84) (10.60.24.10, executor 0, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 73.0 in stage 5.0 (TID 85) (10.60.24.13, executor 1, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 74.0 in stage 5.0 (TID 86) (10.60.24.10, executor 0, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 75.0 in stage 5.0 (TID 87) (10.60.24.13, executor 1, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 76.0 in stage 5.0 (TID 88) (10.60.24.10, executor 0, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 77.0 in stage 5.0 (TID 89) (10.60.24.13, executor 1, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 78.0 in stage 5.0 (TID 90) (10.60.24.10, executor 0, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 79.0 in stage 5.0 (TID 91) (10.60.24.13, executor 1, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 80.0 in stage 5.0 (TID 92) (10.60.24.10, executor 0, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 81.0 in stage 5.0 (TID 93) (10.60.24.13, executor 1, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 82.0 in stage 5.0 (TID 94) (10.60.24.10, executor 0, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 83.0 in stage 5.0 (TID 95) (10.60.24.13, executor 1, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 84.0 in stage 5.0 (TID 96) (10.60.24.10, executor 0, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 85.0 in stage 5.0 (TID 97) (10.60.24.13, executor 1, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 86.0 in stage 5.0 (TID 98) (10.60.24.10, executor 0, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 87.0 in stage 5.0 (TID 99) (10.60.24.13, executor 1, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 88.0 in stage 5.0 (TID 100) (10.60.24.10, executor 0, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 89.0 in stage 5.0 (TID 101) (10.60.24.13, executor 1, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 90.0 in stage 5.0 (TID 102) (10.60.24.10, executor 0, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 91.0 in stage 5.0 (TID 103) (10.60.24.13, executor 1, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 92.0 in stage 5.0 (TID 104) (10.60.24.10, executor 0, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 93.0 in stage 5.0 (TID 105) (10.60.24.13, executor 1, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 94.0 in stage 5.0 (TID 106) (10.60.24.10, executor 0, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 95.0 in stage 5.0 (TID 107) (10.60.24.13, executor 1, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 96.0 in stage 5.0 (TID 108) (10.60.24.10, executor 0, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 97.0 in stage 5.0 (TID 109) (10.60.24.13, executor 1, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 98.0 in stage 5.0 (TID 110) (10.60.24.10, executor 0, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 99.0 in stage 5.0 (TID 111) (10.60.24.13, executor 1, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 100.0 in stage 5.0 (TID 112) (10.60.24.10, executor 0, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 101.0 in stage 5.0 (TID 113) (10.60.24.13, executor 1, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 102.0 in stage 5.0 (TID 114) (10.60.24.10, executor 0, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 103.0 in stage 5.0 (TID 115) (10.60.24.13, executor 1, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 104.0 in stage 5.0 (TID 116) (10.60.24.10, executor 0, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 105.0 in stage 5.0 (TID 117) (10.60.24.13, executor 1, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 106.0 in stage 5.0 (TID 118) (10.60.24.10, executor 0, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 107.0 in stage 5.0 (TID 119) (10.60.24.13, executor 1, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 108.0 in stage 5.0 (TID 120) (10.60.24.10, executor 0, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 109.0 in stage 5.0 (TID 121) (10.60.24.13, executor 1, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 110.0 in stage 5.0 (TID 122) (10.60.24.10, executor 0, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 111.0 in stage 5.0 (TID 123) (10.60.24.13, executor 1, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 112.0 in stage 5.0 (TID 124) (10.60.24.10, executor 0, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 113.0 in stage 5.0 (TID 125) (10.60.24.13, executor 1, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 114.0 in stage 5.0 (TID 126) (10.60.24.10, executor 0, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 115.0 in stage 5.0 (TID 127) (10.60.24.13, executor 1, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 116.0 in stage 5.0 (TID 128) (10.60.24.10, executor 0, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 117.0 in stage 5.0 (TID 129) (10.60.24.13, executor 1, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 118.0 in stage 5.0 (TID 130) (10.60.24.10, executor 0, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 119.0 in stage 5.0 (TID 131) (10.60.24.13, executor 1, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 120.0 in stage 5.0 (TID 132) (10.60.24.10, executor 0, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 121.0 in stage 5.0 (TID 133) (10.60.24.13, executor 1, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 122.0 in stage 5.0 (TID 134) (10.60.24.10, executor 0, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 123.0 in stage 5.0 (TID 135) (10.60.24.13, executor 1, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 124.0 in stage 5.0 (TID 136) (10.60.24.10, executor 0, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 125.0 in stage 5.0 (TID 137) (10.60.24.13, executor 1, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 126.0 in stage 5.0 (TID 138) (10.60.24.10, executor 0, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 127.0 in stage 5.0 (TID 139) (10.60.24.13, executor 1, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 128.0 in stage 5.0 (TID 140) (10.60.24.10, executor 0, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 129.0 in stage 5.0 (TID 141) (10.60.24.13, executor 1, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 130.0 in stage 5.0 (TID 142) (10.60.24.10, executor 0, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 131.0 in stage 5.0 (TID 143) (10.60.24.13, executor 1, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 132.0 in stage 5.0 (TID 144) (10.60.24.10, executor 0, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 133.0 in stage 5.0 (TID 145) (10.60.24.13, executor 1, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 134.0 in stage 5.0 (TID 146) (10.60.24.10, executor 0, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 135.0 in stage 5.0 (TID 147) (10.60.24.13, executor 1, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 136.0 in stage 5.0 (TID 148) (10.60.24.10, executor 0, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 137.0 in stage 5.0 (TID 149) (10.60.24.13, executor 1, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 138.0 in stage 5.0 (TID 150) (10.60.24.10, executor 0, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 139.0 in stage 5.0 (TID 151) (10.60.24.13, executor 1, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 140.0 in stage 5.0 (TID 152) (10.60.24.10, executor 0, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 141.0 in stage 5.0 (TID 153) (10.60.24.13, executor 1, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 142.0 in stage 5.0 (TID 154) (10.60.24.10, executor 0, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 143.0 in stage 5.0 (TID 155) (10.60.24.13, executor 1, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 144.0 in stage 5.0 (TID 156) (10.60.24.10, executor 0, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 145.0 in stage 5.0 (TID 157) (10.60.24.13, executor 1, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 146.0 in stage 5.0 (TID 158) (10.60.24.10, executor 0, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 147.0 in stage 5.0 (TID 159) (10.60.24.13, executor 1, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 148.0 in stage 5.0 (TID 160) (10.60.24.10, executor 0, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 149.0 in stage 5.0 (TID 161) (10.60.24.13, executor 1, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 150.0 in stage 5.0 (TID 162) (10.60.24.10, executor 0, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 151.0 in stage 5.0 (TID 163) (10.60.24.13, executor 1, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 152.0 in stage 5.0 (TID 164) (10.60.24.10, executor 0, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 153.0 in stage 5.0 (TID 165) (10.60.24.13, executor 1, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 154.0 in stage 5.0 (TID 166) (10.60.24.10, executor 0, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 155.0 in stage 5.0 (TID 167) (10.60.24.13, executor 1, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 156.0 in stage 5.0 (TID 168) (10.60.24.10, executor 0, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 157.0 in stage 5.0 (TID 169) (10.60.24.13, executor 1, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 158.0 in stage 5.0 (TID 170) (10.60.24.10, executor 0, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 159.0 in stage 5.0 (TID 171) (10.60.24.13, executor 1, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 160.0 in stage 5.0 (TID 172) (10.60.24.10, executor 0, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 161.0 in stage 5.0 (TID 173) (10.60.24.13, executor 1, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 162.0 in stage 5.0 (TID 174) (10.60.24.10, executor 0, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 163.0 in stage 5.0 (TID 175) (10.60.24.13, executor 1, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 164.0 in stage 5.0 (TID 176) (10.60.24.10, executor 0, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 165.0 in stage 5.0 (TID 177) (10.60.24.13, executor 1, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 166.0 in stage 5.0 (TID 178) (10.60.24.10, executor 0, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 167.0 in stage 5.0 (TID 179) (10.60.24.13, executor 1, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 168.0 in stage 5.0 (TID 180) (10.60.24.10, executor 0, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 169.0 in stage 5.0 (TID 181) (10.60.24.13, executor 1, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 170.0 in stage 5.0 (TID 182) (10.60.24.10, executor 0, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 171.0 in stage 5.0 (TID 183) (10.60.24.13, executor 1, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 172.0 in stage 5.0 (TID 184) (10.60.24.10, executor 0, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 173.0 in stage 5.0 (TID 185) (10.60.24.13, executor 1, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 174.0 in stage 5.0 (TID 186) (10.60.24.10, executor 0, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 175.0 in stage 5.0 (TID 187) (10.60.24.13, executor 1, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 176.0 in stage 5.0 (TID 188) (10.60.24.10, executor 0, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 177.0 in stage 5.0 (TID 189) (10.60.24.13, executor 1, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 178.0 in stage 5.0 (TID 190) (10.60.24.10, executor 0, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 179.0 in stage 5.0 (TID 191) (10.60.24.13, executor 1, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 180.0 in stage 5.0 (TID 192) (10.60.24.10, executor 0, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 181.0 in stage 5.0 (TID 193) (10.60.24.13, executor 1, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 182.0 in stage 5.0 (TID 194) (10.60.24.10, executor 0, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 183.0 in stage 5.0 (TID 195) (10.60.24.13, executor 1, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 184.0 in stage 5.0 (TID 196) (10.60.24.10, executor 0, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 185.0 in stage 5.0 (TID 197) (10.60.24.13, executor 1, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 186.0 in stage 5.0 (TID 198) (10.60.24.10, executor 0, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 187.0 in stage 5.0 (TID 199) (10.60.24.13, executor 1, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 188.0 in stage 5.0 (TID 200) (10.60.24.10, executor 0, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 189.0 in stage 5.0 (TID 201) (10.60.24.13, executor 1, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 190.0 in stage 5.0 (TID 202) (10.60.24.10, executor 0, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 191.0 in stage 5.0 (TID 203) (10.60.24.13, executor 1, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 126.5 KiB, free 750.7 GiB) 23/06/28 07:48:47 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 19.9 KiB, free 750.7 GiB) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on 10.60.24.15:40507 (size: 19.9 KiB, free: 750.7 GiB) 23/06/28 07:48:47 INFO SparkContext: Created broadcast 13 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on 10.60.24.10:41041 (size: 19.9 KiB, free: 487.7 GiB) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on 10.60.24.13:44035 (size: 19.9 KiB, free: 487.7 GiB) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 10.60.24.10:41041 (size: 13.6 KiB, free: 487.7 GiB) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 16.0 in stage 5.0 (TID 28) in 277 ms on 10.60.24.10 (executor 0) (1/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 10.0 in stage 5.0 (TID 22) in 278 ms on 10.60.24.10 (executor 0) (2/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 14.0 in stage 5.0 (TID 26) in 279 ms on 10.60.24.10 (executor 0) (3/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 58.0 in stage 5.0 (TID 70) in 275 ms on 10.60.24.10 (executor 0) (4/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 42.0 in stage 5.0 (TID 54) in 276 ms on 10.60.24.10 (executor 0) (5/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 170.0 in stage 5.0 (TID 182) in 263 ms on 10.60.24.10 (executor 0) (6/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 114.0 in stage 5.0 (TID 126) in 268 ms on 10.60.24.10 (executor 0) (7/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 150.0 in stage 5.0 (TID 162) in 266 ms on 10.60.24.10 (executor 0) (8/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 38.0 in stage 5.0 (TID 50) in 278 ms on 10.60.24.10 (executor 0) (9/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 144.0 in stage 5.0 (TID 156) in 267 ms on 10.60.24.10 (executor 0) (10/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 70.0 in stage 5.0 (TID 82) in 276 ms on 10.60.24.10 (executor 0) (11/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 64.0 in stage 5.0 (TID 76) in 276 ms on 10.60.24.10 (executor 0) (12/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 20.0 in stage 5.0 (TID 32) in 289 ms on 10.60.24.10 (executor 0) (13/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 8.0 in stage 5.0 (TID 20) in 291 ms on 10.60.24.10 (executor 0) (14/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 140.0 in stage 5.0 (TID 152) in 276 ms on 10.60.24.10 (executor 0) (15/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 126.0 in stage 5.0 (TID 138) in 277 ms on 10.60.24.10 (executor 0) (16/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 174.0 in stage 5.0 (TID 186) in 273 ms on 10.60.24.10 (executor 0) (17/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 22.0 in stage 5.0 (TID 34) in 293 ms on 10.60.24.10 (executor 0) (18/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 66.0 in stage 5.0 (TID 78) in 288 ms on 10.60.24.10 (executor 0) (19/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 34.0 in stage 5.0 (TID 46) in 291 ms on 10.60.24.10 (executor 0) (20/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 162.0 in stage 5.0 (TID 174) in 278 ms on 10.60.24.10 (executor 0) (21/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 28.0 in stage 5.0 (TID 40) in 294 ms on 10.60.24.10 (executor 0) (22/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 142.0 in stage 5.0 (TID 154) in 282 ms on 10.60.24.10 (executor 0) (23/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 86.0 in stage 5.0 (TID 98) in 288 ms on 10.60.24.10 (executor 0) (24/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 80.0 in stage 5.0 (TID 92) in 289 ms on 10.60.24.10 (executor 0) (25/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 160.0 in stage 5.0 (TID 172) in 284 ms on 10.60.24.10 (executor 0) (26/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 184.0 in stage 5.0 (TID 196) in 282 ms on 10.60.24.10 (executor 0) (27/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 110.0 in stage 5.0 (TID 122) in 292 ms on 10.60.24.10 (executor 0) (28/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 106.0 in stage 5.0 (TID 118) in 295 ms on 10.60.24.10 (executor 0) (29/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 172.0 in stage 5.0 (TID 184) in 289 ms on 10.60.24.10 (executor 0) (30/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 118.0 in stage 5.0 (TID 130) in 295 ms on 10.60.24.10 (executor 0) (31/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 82.0 in stage 5.0 (TID 94) in 301 ms on 10.60.24.10 (executor 0) (32/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 136.0 in stage 5.0 (TID 148) in 295 ms on 10.60.24.10 (executor 0) (33/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 94.0 in stage 5.0 (TID 106) in 300 ms on 10.60.24.10 (executor 0) (34/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 72.0 in stage 5.0 (TID 84) in 304 ms on 10.60.24.10 (executor 0) (35/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 180.0 in stage 5.0 (TID 192) in 306 ms on 10.60.24.10 (executor 0) (36/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 134.0 in stage 5.0 (TID 146) in 312 ms on 10.60.24.10 (executor 0) (37/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 24.0 in stage 5.0 (TID 36) in 325 ms on 10.60.24.10 (executor 0) (38/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 168.0 in stage 5.0 (TID 180) in 308 ms on 10.60.24.10 (executor 0) (39/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 96.0 in stage 5.0 (TID 108) in 317 ms on 10.60.24.10 (executor 0) (40/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 104.0 in stage 5.0 (TID 116) in 315 ms on 10.60.24.10 (executor 0) (41/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 112.0 in stage 5.0 (TID 124) in 314 ms on 10.60.24.10 (executor 0) (42/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 56.0 in stage 5.0 (TID 68) in 322 ms on 10.60.24.10 (executor 0) (43/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 6.0 in stage 5.0 (TID 18) in 329 ms on 10.60.24.10 (executor 0) (44/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 14) in 330 ms on 10.60.24.10 (executor 0) (45/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 90.0 in stage 5.0 (TID 102) in 319 ms on 10.60.24.10 (executor 0) (46/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 12) in 331 ms on 10.60.24.10 (executor 0) (47/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 146.0 in stage 5.0 (TID 158) in 313 ms on 10.60.24.10 (executor 0) (48/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 156.0 in stage 5.0 (TID 168) in 314 ms on 10.60.24.10 (executor 0) (49/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 12.0 in stage 5.0 (TID 24) in 331 ms on 10.60.24.10 (executor 0) (50/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 120.0 in stage 5.0 (TID 132) in 317 ms on 10.60.24.10 (executor 0) (51/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 78.0 in stage 5.0 (TID 90) in 324 ms on 10.60.24.10 (executor 0) (52/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 74.0 in stage 5.0 (TID 86) in 325 ms on 10.60.24.10 (executor 0) (53/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 182.0 in stage 5.0 (TID 194) in 314 ms on 10.60.24.10 (executor 0) (54/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 130.0 in stage 5.0 (TID 142) in 319 ms on 10.60.24.10 (executor 0) (55/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 128.0 in stage 5.0 (TID 140) in 319 ms on 10.60.24.10 (executor 0) (56/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 152.0 in stage 5.0 (TID 164) in 317 ms on 10.60.24.10 (executor 0) (57/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 44.0 in stage 5.0 (TID 56) in 332 ms on 10.60.24.10 (executor 0) (58/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 164.0 in stage 5.0 (TID 176) in 319 ms on 10.60.24.10 (executor 0) (59/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 178.0 in stage 5.0 (TID 190) in 318 ms on 10.60.24.10 (executor 0) (60/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 122.0 in stage 5.0 (TID 134) in 323 ms on 10.60.24.10 (executor 0) (61/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 166.0 in stage 5.0 (TID 178) in 319 ms on 10.60.24.10 (executor 0) (62/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 100.0 in stage 5.0 (TID 112) in 326 ms on 10.60.24.10 (executor 0) (63/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 176.0 in stage 5.0 (TID 188) in 319 ms on 10.60.24.10 (executor 0) (64/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 108.0 in stage 5.0 (TID 120) in 326 ms on 10.60.24.10 (executor 0) (65/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 54.0 in stage 5.0 (TID 66) in 333 ms on 10.60.24.10 (executor 0) (66/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 188.0 in stage 5.0 (TID 200) in 318 ms on 10.60.24.10 (executor 0) (67/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 40.0 in stage 5.0 (TID 52) in 334 ms on 10.60.24.10 (executor 0) (68/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 158.0 in stage 5.0 (TID 170) in 322 ms on 10.60.24.10 (executor 0) (69/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 148.0 in stage 5.0 (TID 160) in 323 ms on 10.60.24.10 (executor 0) (70/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 138.0 in stage 5.0 (TID 150) in 325 ms on 10.60.24.10 (executor 0) (71/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 52.0 in stage 5.0 (TID 64) in 335 ms on 10.60.24.10 (executor 0) (72/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 30.0 in stage 5.0 (TID 42) in 337 ms on 10.60.24.10 (executor 0) (73/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 18.0 in stage 5.0 (TID 30) in 340 ms on 10.60.24.10 (executor 0) (74/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 60.0 in stage 5.0 (TID 72) in 335 ms on 10.60.24.10 (executor 0) (75/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 98.0 in stage 5.0 (TID 110) in 329 ms on 10.60.24.10 (executor 0) (76/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 102.0 in stage 5.0 (TID 114) in 330 ms on 10.60.24.10 (executor 0) (77/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 88.0 in stage 5.0 (TID 100) in 332 ms on 10.60.24.10 (executor 0) (78/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 132.0 in stage 5.0 (TID 144) in 327 ms on 10.60.24.10 (executor 0) (79/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 4.0 in stage 5.0 (TID 16) in 342 ms on 10.60.24.10 (executor 0) (80/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 50.0 in stage 5.0 (TID 62) in 336 ms on 10.60.24.10 (executor 0) (81/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 84.0 in stage 5.0 (TID 96) in 333 ms on 10.60.24.10 (executor 0) (82/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 186.0 in stage 5.0 (TID 198) in 322 ms on 10.60.24.10 (executor 0) (83/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 76.0 in stage 5.0 (TID 88) in 335 ms on 10.60.24.10 (executor 0) (84/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 190.0 in stage 5.0 (TID 202) in 322 ms on 10.60.24.10 (executor 0) (85/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 124.0 in stage 5.0 (TID 136) in 328 ms on 10.60.24.10 (executor 0) (86/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 48.0 in stage 5.0 (TID 60) in 339 ms on 10.60.24.10 (executor 0) (87/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 32.0 in stage 5.0 (TID 44) in 340 ms on 10.60.24.10 (executor 0) (88/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 154.0 in stage 5.0 (TID 166) in 327 ms on 10.60.24.10 (executor 0) (89/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 26.0 in stage 5.0 (TID 38) in 341 ms on 10.60.24.10 (executor 0) (90/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 46.0 in stage 5.0 (TID 58) in 340 ms on 10.60.24.10 (executor 0) (91/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 116.0 in stage 5.0 (TID 128) in 331 ms on 10.60.24.10 (executor 0) (92/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 62.0 in stage 5.0 (TID 74) in 339 ms on 10.60.24.10 (executor 0) (93/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 36.0 in stage 5.0 (TID 48) in 342 ms on 10.60.24.10 (executor 0) (94/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 68.0 in stage 5.0 (TID 80) in 339 ms on 10.60.24.10 (executor 0) (95/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 92.0 in stage 5.0 (TID 104) in 335 ms on 10.60.24.10 (executor 0) (96/192) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 10.60.24.13:44035 (size: 13.6 KiB, free: 487.7 GiB) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 171.0 in stage 5.0 (TID 183) in 445 ms on 10.60.24.13 (executor 1) (97/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 123.0 in stage 5.0 (TID 135) in 449 ms on 10.60.24.13 (executor 1) (98/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 131.0 in stage 5.0 (TID 143) in 449 ms on 10.60.24.13 (executor 1) (99/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 173.0 in stage 5.0 (TID 185) in 445 ms on 10.60.24.13 (executor 1) (100/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 65.0 in stage 5.0 (TID 77) in 457 ms on 10.60.24.13 (executor 1) (101/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 47.0 in stage 5.0 (TID 59) in 466 ms on 10.60.24.13 (executor 1) (102/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 127.0 in stage 5.0 (TID 139) in 456 ms on 10.60.24.13 (executor 1) (103/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 103.0 in stage 5.0 (TID 115) in 460 ms on 10.60.24.13 (executor 1) (104/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 159.0 in stage 5.0 (TID 171) in 454 ms on 10.60.24.13 (executor 1) (105/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 129.0 in stage 5.0 (TID 141) in 457 ms on 10.60.24.13 (executor 1) (106/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 113.0 in stage 5.0 (TID 125) in 458 ms on 10.60.24.13 (executor 1) (107/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 5.0 in stage 5.0 (TID 17) in 474 ms on 10.60.24.13 (executor 1) (108/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 167.0 in stage 5.0 (TID 179) in 456 ms on 10.60.24.13 (executor 1) (109/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 109.0 in stage 5.0 (TID 121) in 464 ms on 10.60.24.13 (executor 1) (110/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 51.0 in stage 5.0 (TID 63) in 472 ms on 10.60.24.13 (executor 1) (111/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 21.0 in stage 5.0 (TID 33) in 477 ms on 10.60.24.13 (executor 1) (112/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 143.0 in stage 5.0 (TID 155) in 465 ms on 10.60.24.13 (executor 1) (113/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 133.0 in stage 5.0 (TID 145) in 470 ms on 10.60.24.13 (executor 1) (114/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 185.0 in stage 5.0 (TID 197) in 465 ms on 10.60.24.13 (executor 1) (115/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 25.0 in stage 5.0 (TID 37) in 486 ms on 10.60.24.13 (executor 1) (116/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 15.0 in stage 5.0 (TID 27) in 488 ms on 10.60.24.13 (executor 1) (117/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 53.0 in stage 5.0 (TID 65) in 494 ms on 10.60.24.13 (executor 1) (118/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 75.0 in stage 5.0 (TID 87) in 499 ms on 10.60.24.13 (executor 1) (119/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 119.0 in stage 5.0 (TID 131) in 494 ms on 10.60.24.13 (executor 1) (120/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 139.0 in stage 5.0 (TID 151) in 492 ms on 10.60.24.13 (executor 1) (121/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 13.0 in stage 5.0 (TID 25) in 509 ms on 10.60.24.13 (executor 1) (122/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 147.0 in stage 5.0 (TID 159) in 493 ms on 10.60.24.13 (executor 1) (123/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 79.0 in stage 5.0 (TID 91) in 501 ms on 10.60.24.13 (executor 1) (124/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 83.0 in stage 5.0 (TID 95) in 500 ms on 10.60.24.13 (executor 1) (125/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 27.0 in stage 5.0 (TID 39) in 511 ms on 10.60.24.13 (executor 1) (126/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 177.0 in stage 5.0 (TID 189) in 494 ms on 10.60.24.13 (executor 1) (127/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 7.0 in stage 5.0 (TID 19) in 514 ms on 10.60.24.13 (executor 1) (128/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 29.0 in stage 5.0 (TID 41) in 510 ms on 10.60.24.13 (executor 1) (129/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 61.0 in stage 5.0 (TID 73) in 509 ms on 10.60.24.13 (executor 1) (130/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 175.0 in stage 5.0 (TID 187) in 495 ms on 10.60.24.13 (executor 1) (131/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 57.0 in stage 5.0 (TID 69) in 509 ms on 10.60.24.13 (executor 1) (132/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 105.0 in stage 5.0 (TID 117) in 502 ms on 10.60.24.13 (executor 1) (133/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 91.0 in stage 5.0 (TID 103) in 504 ms on 10.60.24.13 (executor 1) (134/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 145.0 in stage 5.0 (TID 157) in 500 ms on 10.60.24.13 (executor 1) (135/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 17.0 in stage 5.0 (TID 29) in 515 ms on 10.60.24.13 (executor 1) (136/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 89.0 in stage 5.0 (TID 101) in 505 ms on 10.60.24.13 (executor 1) (137/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 85.0 in stage 5.0 (TID 97) in 506 ms on 10.60.24.13 (executor 1) (138/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 37.0 in stage 5.0 (TID 49) in 511 ms on 10.60.24.13 (executor 1) (139/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 137.0 in stage 5.0 (TID 149) in 500 ms on 10.60.24.13 (executor 1) (140/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 35.0 in stage 5.0 (TID 47) in 513 ms on 10.60.24.13 (executor 1) (141/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 11.0 in stage 5.0 (TID 23) in 516 ms on 10.60.24.13 (executor 1) (142/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 149.0 in stage 5.0 (TID 161) in 500 ms on 10.60.24.13 (executor 1) (143/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 169.0 in stage 5.0 (TID 181) in 499 ms on 10.60.24.13 (executor 1) (144/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 135.0 in stage 5.0 (TID 147) in 502 ms on 10.60.24.13 (executor 1) (145/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 141.0 in stage 5.0 (TID 153) in 502 ms on 10.60.24.13 (executor 1) (146/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 9.0 in stage 5.0 (TID 21) in 518 ms on 10.60.24.13 (executor 1) (147/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 163.0 in stage 5.0 (TID 175) in 500 ms on 10.60.24.13 (executor 1) (148/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 117.0 in stage 5.0 (TID 129) in 505 ms on 10.60.24.13 (executor 1) (149/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 33.0 in stage 5.0 (TID 45) in 515 ms on 10.60.24.13 (executor 1) (150/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 183.0 in stage 5.0 (TID 195) in 498 ms on 10.60.24.13 (executor 1) (151/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 153.0 in stage 5.0 (TID 165) in 502 ms on 10.60.24.13 (executor 1) (152/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 97.0 in stage 5.0 (TID 109) in 508 ms on 10.60.24.13 (executor 1) (153/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 69.0 in stage 5.0 (TID 81) in 512 ms on 10.60.24.13 (executor 1) (154/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 151.0 in stage 5.0 (TID 163) in 502 ms on 10.60.24.13 (executor 1) (155/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 161.0 in stage 5.0 (TID 173) in 501 ms on 10.60.24.13 (executor 1) (156/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 71.0 in stage 5.0 (TID 83) in 512 ms on 10.60.24.13 (executor 1) (157/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 15) in 520 ms on 10.60.24.13 (executor 1) (158/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 67.0 in stage 5.0 (TID 79) in 513 ms on 10.60.24.13 (executor 1) (159/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 99.0 in stage 5.0 (TID 111) in 508 ms on 10.60.24.13 (executor 1) (160/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 13) in 521 ms on 10.60.24.13 (executor 1) (161/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 95.0 in stage 5.0 (TID 107) in 510 ms on 10.60.24.13 (executor 1) (162/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 179.0 in stage 5.0 (TID 191) in 501 ms on 10.60.24.13 (executor 1) (163/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 55.0 in stage 5.0 (TID 67) in 515 ms on 10.60.24.13 (executor 1) (164/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 115.0 in stage 5.0 (TID 127) in 507 ms on 10.60.24.13 (executor 1) (165/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 155.0 in stage 5.0 (TID 167) in 504 ms on 10.60.24.13 (executor 1) (166/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 93.0 in stage 5.0 (TID 105) in 510 ms on 10.60.24.13 (executor 1) (167/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 107.0 in stage 5.0 (TID 119) in 509 ms on 10.60.24.13 (executor 1) (168/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 43.0 in stage 5.0 (TID 55) in 517 ms on 10.60.24.13 (executor 1) (169/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 77.0 in stage 5.0 (TID 89) in 514 ms on 10.60.24.13 (executor 1) (170/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 181.0 in stage 5.0 (TID 193) in 502 ms on 10.60.24.13 (executor 1) (171/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 111.0 in stage 5.0 (TID 123) in 509 ms on 10.60.24.13 (executor 1) (172/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 87.0 in stage 5.0 (TID 99) in 512 ms on 10.60.24.13 (executor 1) (173/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 63.0 in stage 5.0 (TID 75) in 515 ms on 10.60.24.13 (executor 1) (174/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 81.0 in stage 5.0 (TID 93) in 512 ms on 10.60.24.13 (executor 1) (175/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 31.0 in stage 5.0 (TID 43) in 519 ms on 10.60.24.13 (executor 1) (176/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 165.0 in stage 5.0 (TID 177) in 504 ms on 10.60.24.13 (executor 1) (177/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 39.0 in stage 5.0 (TID 51) in 518 ms on 10.60.24.13 (executor 1) (178/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 41.0 in stage 5.0 (TID 53) in 518 ms on 10.60.24.13 (executor 1) (179/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 45.0 in stage 5.0 (TID 57) in 518 ms on 10.60.24.13 (executor 1) (180/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 73.0 in stage 5.0 (TID 85) in 515 ms on 10.60.24.13 (executor 1) (181/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 189.0 in stage 5.0 (TID 201) in 502 ms on 10.60.24.13 (executor 1) (182/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 49.0 in stage 5.0 (TID 61) in 518 ms on 10.60.24.13 (executor 1) (183/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 23.0 in stage 5.0 (TID 35) in 522 ms on 10.60.24.13 (executor 1) (184/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 101.0 in stage 5.0 (TID 113) in 512 ms on 10.60.24.13 (executor 1) (185/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 125.0 in stage 5.0 (TID 137) in 509 ms on 10.60.24.13 (executor 1) (186/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 59.0 in stage 5.0 (TID 71) in 518 ms on 10.60.24.13 (executor 1) (187/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 187.0 in stage 5.0 (TID 199) in 503 ms on 10.60.24.13 (executor 1) (188/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 19.0 in stage 5.0 (TID 31) in 522 ms on 10.60.24.13 (executor 1) (189/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 121.0 in stage 5.0 (TID 133) in 509 ms on 10.60.24.13 (executor 1) (190/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 157.0 in stage 5.0 (TID 169) in 507 ms on 10.60.24.13 (executor 1) (191/192) 23/06/28 07:48:47 INFO TaskSetManager: Finished task 191.0 in stage 5.0 (TID 203) in 782 ms on 10.60.24.13 (executor 1) (192/192) 23/06/28 07:48:47 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:47 INFO DAGScheduler: ShuffleMapStage 5 (repartition at DeltaFileOperations.scala:251) finished in 0.812 s 23/06/28 07:48:47 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:47 INFO DAGScheduler: running: Set() 23/06/28 07:48:47 INFO DAGScheduler: waiting: Set(ShuffleMapStage 6, ShuffleMapStage 7) 23/06/28 07:48:47 INFO DAGScheduler: failed: Set() 23/06/28 07:48:47 INFO DAGScheduler: Submitting ShuffleMapStage 6 (MapPartitionsRDD[33] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:47 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:47 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:47 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:47 INFO DAGScheduler: Submitting 200 missing tasks from ShuffleMapStage 6 (MapPartitionsRDD[33] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:47 INFO TaskSchedulerImpl: Adding task set 6.0 with 200 tasks resource profile 0 23/06/28 07:48:47 INFO FairSchedulableBuilder: Added task set TaskSet_6.0 tasks to pool 7670323585580043917 23/06/28 07:48:47 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 204) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 205) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 206) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 207) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 4.0 in stage 6.0 (TID 208) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 5.0 in stage 6.0 (TID 209) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 6.0 in stage 6.0 (TID 210) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 7.0 in stage 6.0 (TID 211) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 8.0 in stage 6.0 (TID 212) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 9.0 in stage 6.0 (TID 213) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 10.0 in stage 6.0 (TID 214) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 11.0 in stage 6.0 (TID 215) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 12.0 in stage 6.0 (TID 216) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 13.0 in stage 6.0 (TID 217) (10.60.24.10, executor 0, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 14.0 in stage 6.0 (TID 218) (10.60.24.13, executor 1, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 15.0 in stage 6.0 (TID 219) (10.60.24.10, executor 0, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 16.0 in stage 6.0 (TID 220) (10.60.24.13, executor 1, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 17.0 in stage 6.0 (TID 221) (10.60.24.10, executor 0, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 18.0 in stage 6.0 (TID 222) (10.60.24.13, executor 1, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 19.0 in stage 6.0 (TID 223) (10.60.24.10, executor 0, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 20.0 in stage 6.0 (TID 224) (10.60.24.13, executor 1, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 21.0 in stage 6.0 (TID 225) (10.60.24.10, executor 0, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 22.0 in stage 6.0 (TID 226) (10.60.24.13, executor 1, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 23.0 in stage 6.0 (TID 227) (10.60.24.10, executor 0, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 24.0 in stage 6.0 (TID 228) (10.60.24.13, executor 1, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 25.0 in stage 6.0 (TID 229) (10.60.24.10, executor 0, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 26.0 in stage 6.0 (TID 230) (10.60.24.13, executor 1, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 27.0 in stage 6.0 (TID 231) (10.60.24.10, executor 0, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 28.0 in stage 6.0 (TID 232) (10.60.24.13, executor 1, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 29.0 in stage 6.0 (TID 233) (10.60.24.10, executor 0, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 30.0 in stage 6.0 (TID 234) (10.60.24.13, executor 1, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 31.0 in stage 6.0 (TID 235) (10.60.24.10, executor 0, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 32.0 in stage 6.0 (TID 236) (10.60.24.13, executor 1, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 33.0 in stage 6.0 (TID 237) (10.60.24.10, executor 0, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 34.0 in stage 6.0 (TID 238) (10.60.24.13, executor 1, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 35.0 in stage 6.0 (TID 239) (10.60.24.10, executor 0, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 36.0 in stage 6.0 (TID 240) (10.60.24.13, executor 1, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 37.0 in stage 6.0 (TID 241) (10.60.24.10, executor 0, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 38.0 in stage 6.0 (TID 242) (10.60.24.13, executor 1, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 39.0 in stage 6.0 (TID 243) (10.60.24.10, executor 0, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 40.0 in stage 6.0 (TID 244) (10.60.24.13, executor 1, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 41.0 in stage 6.0 (TID 245) (10.60.24.10, executor 0, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 42.0 in stage 6.0 (TID 246) (10.60.24.13, executor 1, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 43.0 in stage 6.0 (TID 247) (10.60.24.10, executor 0, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 44.0 in stage 6.0 (TID 248) (10.60.24.13, executor 1, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 45.0 in stage 6.0 (TID 249) (10.60.24.10, executor 0, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 46.0 in stage 6.0 (TID 250) (10.60.24.13, executor 1, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 47.0 in stage 6.0 (TID 251) (10.60.24.10, executor 0, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 48.0 in stage 6.0 (TID 252) (10.60.24.13, executor 1, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 49.0 in stage 6.0 (TID 253) (10.60.24.10, executor 0, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 50.0 in stage 6.0 (TID 254) (10.60.24.13, executor 1, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 51.0 in stage 6.0 (TID 255) (10.60.24.10, executor 0, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 52.0 in stage 6.0 (TID 256) (10.60.24.13, executor 1, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 53.0 in stage 6.0 (TID 257) (10.60.24.10, executor 0, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 54.0 in stage 6.0 (TID 258) (10.60.24.13, executor 1, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 55.0 in stage 6.0 (TID 259) (10.60.24.10, executor 0, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 56.0 in stage 6.0 (TID 260) (10.60.24.13, executor 1, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 57.0 in stage 6.0 (TID 261) (10.60.24.10, executor 0, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 58.0 in stage 6.0 (TID 262) (10.60.24.13, executor 1, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 59.0 in stage 6.0 (TID 263) (10.60.24.10, executor 0, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 60.0 in stage 6.0 (TID 264) (10.60.24.13, executor 1, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 61.0 in stage 6.0 (TID 265) (10.60.24.10, executor 0, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 62.0 in stage 6.0 (TID 266) (10.60.24.13, executor 1, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 63.0 in stage 6.0 (TID 267) (10.60.24.10, executor 0, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 64.0 in stage 6.0 (TID 268) (10.60.24.13, executor 1, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 65.0 in stage 6.0 (TID 269) (10.60.24.10, executor 0, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 66.0 in stage 6.0 (TID 270) (10.60.24.13, executor 1, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 67.0 in stage 6.0 (TID 271) (10.60.24.10, executor 0, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 68.0 in stage 6.0 (TID 272) (10.60.24.13, executor 1, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 69.0 in stage 6.0 (TID 273) (10.60.24.10, executor 0, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 70.0 in stage 6.0 (TID 274) (10.60.24.13, executor 1, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 71.0 in stage 6.0 (TID 275) (10.60.24.10, executor 0, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 72.0 in stage 6.0 (TID 276) (10.60.24.13, executor 1, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 73.0 in stage 6.0 (TID 277) (10.60.24.10, executor 0, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 74.0 in stage 6.0 (TID 278) (10.60.24.13, executor 1, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 75.0 in stage 6.0 (TID 279) (10.60.24.10, executor 0, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 76.0 in stage 6.0 (TID 280) (10.60.24.13, executor 1, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 77.0 in stage 6.0 (TID 281) (10.60.24.10, executor 0, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 78.0 in stage 6.0 (TID 282) (10.60.24.13, executor 1, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 79.0 in stage 6.0 (TID 283) (10.60.24.10, executor 0, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 80.0 in stage 6.0 (TID 284) (10.60.24.13, executor 1, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 81.0 in stage 6.0 (TID 285) (10.60.24.10, executor 0, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 82.0 in stage 6.0 (TID 286) (10.60.24.13, executor 1, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 83.0 in stage 6.0 (TID 287) (10.60.24.10, executor 0, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 84.0 in stage 6.0 (TID 288) (10.60.24.13, executor 1, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 85.0 in stage 6.0 (TID 289) (10.60.24.10, executor 0, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 86.0 in stage 6.0 (TID 290) (10.60.24.13, executor 1, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 87.0 in stage 6.0 (TID 291) (10.60.24.10, executor 0, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 88.0 in stage 6.0 (TID 292) (10.60.24.13, executor 1, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 89.0 in stage 6.0 (TID 293) (10.60.24.10, executor 0, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 90.0 in stage 6.0 (TID 294) (10.60.24.13, executor 1, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 91.0 in stage 6.0 (TID 295) (10.60.24.10, executor 0, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 92.0 in stage 6.0 (TID 296) (10.60.24.13, executor 1, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 93.0 in stage 6.0 (TID 297) (10.60.24.10, executor 0, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 94.0 in stage 6.0 (TID 298) (10.60.24.13, executor 1, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 95.0 in stage 6.0 (TID 299) (10.60.24.10, executor 0, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 96.0 in stage 6.0 (TID 300) (10.60.24.13, executor 1, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 97.0 in stage 6.0 (TID 301) (10.60.24.10, executor 0, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 98.0 in stage 6.0 (TID 302) (10.60.24.13, executor 1, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 99.0 in stage 6.0 (TID 303) (10.60.24.10, executor 0, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 100.0 in stage 6.0 (TID 304) (10.60.24.13, executor 1, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 101.0 in stage 6.0 (TID 305) (10.60.24.10, executor 0, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 102.0 in stage 6.0 (TID 306) (10.60.24.13, executor 1, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 103.0 in stage 6.0 (TID 307) (10.60.24.10, executor 0, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 104.0 in stage 6.0 (TID 308) (10.60.24.13, executor 1, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 105.0 in stage 6.0 (TID 309) (10.60.24.10, executor 0, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 106.0 in stage 6.0 (TID 310) (10.60.24.13, executor 1, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 107.0 in stage 6.0 (TID 311) (10.60.24.10, executor 0, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 108.0 in stage 6.0 (TID 312) (10.60.24.13, executor 1, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 109.0 in stage 6.0 (TID 313) (10.60.24.10, executor 0, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 110.0 in stage 6.0 (TID 314) (10.60.24.13, executor 1, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 111.0 in stage 6.0 (TID 315) (10.60.24.10, executor 0, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 112.0 in stage 6.0 (TID 316) (10.60.24.13, executor 1, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 113.0 in stage 6.0 (TID 317) (10.60.24.10, executor 0, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 114.0 in stage 6.0 (TID 318) (10.60.24.13, executor 1, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 115.0 in stage 6.0 (TID 319) (10.60.24.10, executor 0, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 116.0 in stage 6.0 (TID 320) (10.60.24.13, executor 1, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 117.0 in stage 6.0 (TID 321) (10.60.24.10, executor 0, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 118.0 in stage 6.0 (TID 322) (10.60.24.13, executor 1, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 119.0 in stage 6.0 (TID 323) (10.60.24.10, executor 0, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 120.0 in stage 6.0 (TID 324) (10.60.24.13, executor 1, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 121.0 in stage 6.0 (TID 325) (10.60.24.10, executor 0, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 122.0 in stage 6.0 (TID 326) (10.60.24.13, executor 1, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 123.0 in stage 6.0 (TID 327) (10.60.24.10, executor 0, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 124.0 in stage 6.0 (TID 328) (10.60.24.13, executor 1, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 125.0 in stage 6.0 (TID 329) (10.60.24.10, executor 0, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 126.0 in stage 6.0 (TID 330) (10.60.24.13, executor 1, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 127.0 in stage 6.0 (TID 331) (10.60.24.10, executor 0, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 128.0 in stage 6.0 (TID 332) (10.60.24.13, executor 1, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 129.0 in stage 6.0 (TID 333) (10.60.24.10, executor 0, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 130.0 in stage 6.0 (TID 334) (10.60.24.13, executor 1, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 131.0 in stage 6.0 (TID 335) (10.60.24.10, executor 0, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 132.0 in stage 6.0 (TID 336) (10.60.24.13, executor 1, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 133.0 in stage 6.0 (TID 337) (10.60.24.10, executor 0, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 134.0 in stage 6.0 (TID 338) (10.60.24.13, executor 1, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 135.0 in stage 6.0 (TID 339) (10.60.24.10, executor 0, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 136.0 in stage 6.0 (TID 340) (10.60.24.13, executor 1, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 137.0 in stage 6.0 (TID 341) (10.60.24.10, executor 0, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 138.0 in stage 6.0 (TID 342) (10.60.24.13, executor 1, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 139.0 in stage 6.0 (TID 343) (10.60.24.10, executor 0, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 140.0 in stage 6.0 (TID 344) (10.60.24.13, executor 1, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 141.0 in stage 6.0 (TID 345) (10.60.24.10, executor 0, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 142.0 in stage 6.0 (TID 346) (10.60.24.13, executor 1, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 143.0 in stage 6.0 (TID 347) (10.60.24.10, executor 0, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 144.0 in stage 6.0 (TID 348) (10.60.24.13, executor 1, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 145.0 in stage 6.0 (TID 349) (10.60.24.10, executor 0, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 146.0 in stage 6.0 (TID 350) (10.60.24.13, executor 1, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 147.0 in stage 6.0 (TID 351) (10.60.24.10, executor 0, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 148.0 in stage 6.0 (TID 352) (10.60.24.13, executor 1, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 149.0 in stage 6.0 (TID 353) (10.60.24.10, executor 0, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 150.0 in stage 6.0 (TID 354) (10.60.24.13, executor 1, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 151.0 in stage 6.0 (TID 355) (10.60.24.10, executor 0, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 152.0 in stage 6.0 (TID 356) (10.60.24.13, executor 1, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 153.0 in stage 6.0 (TID 357) (10.60.24.10, executor 0, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 154.0 in stage 6.0 (TID 358) (10.60.24.13, executor 1, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 155.0 in stage 6.0 (TID 359) (10.60.24.10, executor 0, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 156.0 in stage 6.0 (TID 360) (10.60.24.13, executor 1, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 157.0 in stage 6.0 (TID 361) (10.60.24.10, executor 0, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 158.0 in stage 6.0 (TID 362) (10.60.24.13, executor 1, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 159.0 in stage 6.0 (TID 363) (10.60.24.10, executor 0, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 160.0 in stage 6.0 (TID 364) (10.60.24.13, executor 1, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 161.0 in stage 6.0 (TID 365) (10.60.24.10, executor 0, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 162.0 in stage 6.0 (TID 366) (10.60.24.13, executor 1, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 163.0 in stage 6.0 (TID 367) (10.60.24.10, executor 0, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 164.0 in stage 6.0 (TID 368) (10.60.24.13, executor 1, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 165.0 in stage 6.0 (TID 369) (10.60.24.10, executor 0, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 166.0 in stage 6.0 (TID 370) (10.60.24.13, executor 1, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 167.0 in stage 6.0 (TID 371) (10.60.24.10, executor 0, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 168.0 in stage 6.0 (TID 372) (10.60.24.13, executor 1, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 169.0 in stage 6.0 (TID 373) (10.60.24.10, executor 0, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 170.0 in stage 6.0 (TID 374) (10.60.24.13, executor 1, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 171.0 in stage 6.0 (TID 375) (10.60.24.10, executor 0, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 172.0 in stage 6.0 (TID 376) (10.60.24.13, executor 1, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 173.0 in stage 6.0 (TID 377) (10.60.24.10, executor 0, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 174.0 in stage 6.0 (TID 378) (10.60.24.13, executor 1, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 175.0 in stage 6.0 (TID 379) (10.60.24.10, executor 0, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 176.0 in stage 6.0 (TID 380) (10.60.24.13, executor 1, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 177.0 in stage 6.0 (TID 381) (10.60.24.10, executor 0, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 178.0 in stage 6.0 (TID 382) (10.60.24.13, executor 1, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 179.0 in stage 6.0 (TID 383) (10.60.24.10, executor 0, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 180.0 in stage 6.0 (TID 384) (10.60.24.13, executor 1, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 181.0 in stage 6.0 (TID 385) (10.60.24.10, executor 0, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 182.0 in stage 6.0 (TID 386) (10.60.24.13, executor 1, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 183.0 in stage 6.0 (TID 387) (10.60.24.10, executor 0, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 184.0 in stage 6.0 (TID 388) (10.60.24.13, executor 1, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 185.0 in stage 6.0 (TID 389) (10.60.24.10, executor 0, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 186.0 in stage 6.0 (TID 390) (10.60.24.13, executor 1, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 187.0 in stage 6.0 (TID 391) (10.60.24.10, executor 0, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 188.0 in stage 6.0 (TID 392) (10.60.24.13, executor 1, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 189.0 in stage 6.0 (TID 393) (10.60.24.10, executor 0, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 190.0 in stage 6.0 (TID 394) (10.60.24.13, executor 1, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO TaskSetManager: Starting task 191.0 in stage 6.0 (TID 395) (10.60.24.10, executor 0, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:47 INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 146.8 KiB, free 750.7 GiB) 23/06/28 07:48:47 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 29.2 KiB, free 750.7 GiB) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on 10.60.24.15:40507 (size: 29.2 KiB, free: 750.7 GiB) 23/06/28 07:48:47 INFO SparkContext: Created broadcast 14 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on 10.60.24.13:44035 (size: 29.2 KiB, free: 487.7 GiB) 23/06/28 07:48:47 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on 10.60.24.10:41041 (size: 29.2 KiB, free: 487.7 GiB) 23/06/28 07:48:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 10.60.24.10:44288 23/06/28 07:48:48 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 3 to 10.60.24.13:58548 23/06/28 07:48:48 INFO TaskSetManager: Starting task 192.0 in stage 6.0 (TID 396) (10.60.24.10, executor 0, partition 192, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 193.0 in stage 6.0 (TID 397) (10.60.24.10, executor 0, partition 193, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 194.0 in stage 6.0 (TID 398) (10.60.24.10, executor 0, partition 194, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 195.0 in stage 6.0 (TID 399) (10.60.24.10, executor 0, partition 195, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 79.0 in stage 6.0 (TID 283) in 683 ms on 10.60.24.10 (executor 0) (1/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 173.0 in stage 6.0 (TID 377) in 677 ms on 10.60.24.10 (executor 0) (2/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 143.0 in stage 6.0 (TID 347) in 680 ms on 10.60.24.10 (executor 0) (3/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 133.0 in stage 6.0 (TID 337) in 680 ms on 10.60.24.10 (executor 0) (4/200) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 196.0 in stage 6.0 (TID 400) (10.60.24.10, executor 0, partition 196, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 197.0 in stage 6.0 (TID 401) (10.60.24.10, executor 0, partition 197, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 155.0 in stage 6.0 (TID 359) in 678 ms on 10.60.24.10 (executor 0) (5/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 147.0 in stage 6.0 (TID 351) in 679 ms on 10.60.24.10 (executor 0) (6/200) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 198.0 in stage 6.0 (TID 402) (10.60.24.10, executor 0, partition 198, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 199.0 in stage 6.0 (TID 403) (10.60.24.10, executor 0, partition 199, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 149.0 in stage 6.0 (TID 353) in 680 ms on 10.60.24.10 (executor 0) (7/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 17.0 in stage 6.0 (TID 221) in 688 ms on 10.60.24.10 (executor 0) (8/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 91.0 in stage 6.0 (TID 295) in 684 ms on 10.60.24.10 (executor 0) (9/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 83.0 in stage 6.0 (TID 287) in 684 ms on 10.60.24.10 (executor 0) (10/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 53.0 in stage 6.0 (TID 257) in 686 ms on 10.60.24.10 (executor 0) (11/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 99.0 in stage 6.0 (TID 303) in 683 ms on 10.60.24.10 (executor 0) (12/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 105.0 in stage 6.0 (TID 309) in 684 ms on 10.60.24.10 (executor 0) (13/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 29.0 in stage 6.0 (TID 233) in 688 ms on 10.60.24.10 (executor 0) (14/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 121.0 in stage 6.0 (TID 325) in 683 ms on 10.60.24.10 (executor 0) (15/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 69.0 in stage 6.0 (TID 273) in 686 ms on 10.60.24.10 (executor 0) (16/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 93.0 in stage 6.0 (TID 297) in 685 ms on 10.60.24.10 (executor 0) (17/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 45.0 in stage 6.0 (TID 249) in 687 ms on 10.60.24.10 (executor 0) (18/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 63.0 in stage 6.0 (TID 267) in 686 ms on 10.60.24.10 (executor 0) (19/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 179.0 in stage 6.0 (TID 383) in 678 ms on 10.60.24.10 (executor 0) (20/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 167.0 in stage 6.0 (TID 371) in 679 ms on 10.60.24.10 (executor 0) (21/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 161.0 in stage 6.0 (TID 365) in 681 ms on 10.60.24.10 (executor 0) (22/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 137.0 in stage 6.0 (TID 341) in 683 ms on 10.60.24.10 (executor 0) (23/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 9.0 in stage 6.0 (TID 213) in 691 ms on 10.60.24.10 (executor 0) (24/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 109.0 in stage 6.0 (TID 313) in 684 ms on 10.60.24.10 (executor 0) (25/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 15.0 in stage 6.0 (TID 219) in 690 ms on 10.60.24.10 (executor 0) (26/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 111.0 in stage 6.0 (TID 315) in 684 ms on 10.60.24.10 (executor 0) (27/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 131.0 in stage 6.0 (TID 335) in 683 ms on 10.60.24.10 (executor 0) (28/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 47.0 in stage 6.0 (TID 251) in 688 ms on 10.60.24.10 (executor 0) (29/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 73.0 in stage 6.0 (TID 277) in 687 ms on 10.60.24.10 (executor 0) (30/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 11.0 in stage 6.0 (TID 215) in 690 ms on 10.60.24.10 (executor 0) (31/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 57.0 in stage 6.0 (TID 261) in 687 ms on 10.60.24.10 (executor 0) (32/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 113.0 in stage 6.0 (TID 317) in 684 ms on 10.60.24.10 (executor 0) (33/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 81.0 in stage 6.0 (TID 285) in 687 ms on 10.60.24.10 (executor 0) (34/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 85.0 in stage 6.0 (TID 289) in 687 ms on 10.60.24.10 (executor 0) (35/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 101.0 in stage 6.0 (TID 305) in 686 ms on 10.60.24.10 (executor 0) (36/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 19.0 in stage 6.0 (TID 223) in 691 ms on 10.60.24.10 (executor 0) (37/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 95.0 in stage 6.0 (TID 299) in 686 ms on 10.60.24.10 (executor 0) (38/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 7.0 in stage 6.0 (TID 211) in 692 ms on 10.60.24.10 (executor 0) (39/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 41.0 in stage 6.0 (TID 245) in 689 ms on 10.60.24.10 (executor 0) (40/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 171.0 in stage 6.0 (TID 375) in 681 ms on 10.60.24.10 (executor 0) (41/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 25.0 in stage 6.0 (TID 229) in 690 ms on 10.60.24.10 (executor 0) (42/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 135.0 in stage 6.0 (TID 339) in 684 ms on 10.60.24.10 (executor 0) (43/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 169.0 in stage 6.0 (TID 373) in 681 ms on 10.60.24.10 (executor 0) (44/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 139.0 in stage 6.0 (TID 343) in 684 ms on 10.60.24.10 (executor 0) (45/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 119.0 in stage 6.0 (TID 323) in 686 ms on 10.60.24.10 (executor 0) (46/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 65.0 in stage 6.0 (TID 269) in 689 ms on 10.60.24.10 (executor 0) (47/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 103.0 in stage 6.0 (TID 307) in 687 ms on 10.60.24.10 (executor 0) (48/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 181.0 in stage 6.0 (TID 385) in 681 ms on 10.60.24.10 (executor 0) (49/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 191.0 in stage 6.0 (TID 395) in 681 ms on 10.60.24.10 (executor 0) (50/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 67.0 in stage 6.0 (TID 271) in 689 ms on 10.60.24.10 (executor 0) (51/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 165.0 in stage 6.0 (TID 369) in 682 ms on 10.60.24.10 (executor 0) (52/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 37.0 in stage 6.0 (TID 241) in 691 ms on 10.60.24.10 (executor 0) (53/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 87.0 in stage 6.0 (TID 291) in 688 ms on 10.60.24.10 (executor 0) (54/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 55.0 in stage 6.0 (TID 259) in 690 ms on 10.60.24.10 (executor 0) (55/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 145.0 in stage 6.0 (TID 349) in 685 ms on 10.60.24.10 (executor 0) (56/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 205) in 694 ms on 10.60.24.10 (executor 0) (57/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 43.0 in stage 6.0 (TID 247) in 691 ms on 10.60.24.10 (executor 0) (58/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 39.0 in stage 6.0 (TID 243) in 693 ms on 10.60.24.10 (executor 0) (59/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 31.0 in stage 6.0 (TID 235) in 693 ms on 10.60.24.10 (executor 0) (60/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 117.0 in stage 6.0 (TID 321) in 688 ms on 10.60.24.10 (executor 0) (61/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 21.0 in stage 6.0 (TID 225) in 694 ms on 10.60.24.10 (executor 0) (62/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 207) in 695 ms on 10.60.24.10 (executor 0) (63/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 187.0 in stage 6.0 (TID 391) in 684 ms on 10.60.24.10 (executor 0) (64/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 61.0 in stage 6.0 (TID 265) in 692 ms on 10.60.24.10 (executor 0) (65/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 141.0 in stage 6.0 (TID 345) in 689 ms on 10.60.24.10 (executor 0) (66/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 75.0 in stage 6.0 (TID 279) in 692 ms on 10.60.24.10 (executor 0) (67/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 77.0 in stage 6.0 (TID 281) in 692 ms on 10.60.24.10 (executor 0) (68/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 123.0 in stage 6.0 (TID 327) in 690 ms on 10.60.24.10 (executor 0) (69/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 5.0 in stage 6.0 (TID 209) in 697 ms on 10.60.24.10 (executor 0) (70/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 157.0 in stage 6.0 (TID 361) in 687 ms on 10.60.24.10 (executor 0) (71/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 89.0 in stage 6.0 (TID 293) in 692 ms on 10.60.24.10 (executor 0) (72/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 115.0 in stage 6.0 (TID 319) in 691 ms on 10.60.24.10 (executor 0) (73/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 59.0 in stage 6.0 (TID 263) in 694 ms on 10.60.24.10 (executor 0) (74/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 49.0 in stage 6.0 (TID 253) in 695 ms on 10.60.24.10 (executor 0) (75/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 185.0 in stage 6.0 (TID 389) in 686 ms on 10.60.24.10 (executor 0) (76/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 183.0 in stage 6.0 (TID 387) in 686 ms on 10.60.24.10 (executor 0) (77/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 175.0 in stage 6.0 (TID 379) in 689 ms on 10.60.24.10 (executor 0) (78/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 151.0 in stage 6.0 (TID 355) in 691 ms on 10.60.24.10 (executor 0) (79/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 13.0 in stage 6.0 (TID 217) in 699 ms on 10.60.24.10 (executor 0) (80/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 127.0 in stage 6.0 (TID 331) in 692 ms on 10.60.24.10 (executor 0) (81/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 97.0 in stage 6.0 (TID 301) in 695 ms on 10.60.24.10 (executor 0) (82/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 177.0 in stage 6.0 (TID 381) in 689 ms on 10.60.24.10 (executor 0) (83/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 71.0 in stage 6.0 (TID 275) in 697 ms on 10.60.24.10 (executor 0) (84/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 163.0 in stage 6.0 (TID 367) in 690 ms on 10.60.24.10 (executor 0) (85/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 125.0 in stage 6.0 (TID 329) in 694 ms on 10.60.24.10 (executor 0) (86/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 153.0 in stage 6.0 (TID 357) in 692 ms on 10.60.24.10 (executor 0) (87/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 23.0 in stage 6.0 (TID 227) in 700 ms on 10.60.24.10 (executor 0) (88/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 51.0 in stage 6.0 (TID 255) in 699 ms on 10.60.24.10 (executor 0) (89/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 189.0 in stage 6.0 (TID 393) in 690 ms on 10.60.24.10 (executor 0) (90/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 33.0 in stage 6.0 (TID 237) in 700 ms on 10.60.24.10 (executor 0) (91/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 107.0 in stage 6.0 (TID 311) in 695 ms on 10.60.24.10 (executor 0) (92/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 35.0 in stage 6.0 (TID 239) in 700 ms on 10.60.24.10 (executor 0) (93/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 27.0 in stage 6.0 (TID 231) in 701 ms on 10.60.24.10 (executor 0) (94/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 129.0 in stage 6.0 (TID 333) in 695 ms on 10.60.24.10 (executor 0) (95/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 159.0 in stage 6.0 (TID 363) in 693 ms on 10.60.24.10 (executor 0) (96/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 196.0 in stage 6.0 (TID 400) in 20 ms on 10.60.24.10 (executor 0) (97/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 199.0 in stage 6.0 (TID 403) in 19 ms on 10.60.24.10 (executor 0) (98/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 198.0 in stage 6.0 (TID 402) in 19 ms on 10.60.24.10 (executor 0) (99/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 195.0 in stage 6.0 (TID 399) in 22 ms on 10.60.24.10 (executor 0) (100/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 197.0 in stage 6.0 (TID 401) in 21 ms on 10.60.24.10 (executor 0) (101/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 192.0 in stage 6.0 (TID 396) in 23 ms on 10.60.24.10 (executor 0) (102/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 193.0 in stage 6.0 (TID 397) in 22 ms on 10.60.24.10 (executor 0) (103/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 194.0 in stage 6.0 (TID 398) in 23 ms on 10.60.24.10 (executor 0) (104/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 152.0 in stage 6.0 (TID 356) in 756 ms on 10.60.24.13 (executor 1) (105/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 150.0 in stage 6.0 (TID 354) in 756 ms on 10.60.24.13 (executor 1) (106/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 124.0 in stage 6.0 (TID 328) in 758 ms on 10.60.24.13 (executor 1) (107/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 114.0 in stage 6.0 (TID 318) in 758 ms on 10.60.24.13 (executor 1) (108/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 116.0 in stage 6.0 (TID 320) in 758 ms on 10.60.24.13 (executor 1) (109/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 6.0 in stage 6.0 (TID 210) in 765 ms on 10.60.24.13 (executor 1) (110/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 10.0 in stage 6.0 (TID 214) in 766 ms on 10.60.24.13 (executor 1) (111/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 78.0 in stage 6.0 (TID 282) in 761 ms on 10.60.24.13 (executor 1) (112/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 14.0 in stage 6.0 (TID 218) in 765 ms on 10.60.24.13 (executor 1) (113/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 180.0 in stage 6.0 (TID 384) in 754 ms on 10.60.24.13 (executor 1) (114/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 96.0 in stage 6.0 (TID 300) in 760 ms on 10.60.24.13 (executor 1) (115/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 178.0 in stage 6.0 (TID 382) in 754 ms on 10.60.24.13 (executor 1) (116/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 74.0 in stage 6.0 (TID 278) in 761 ms on 10.60.24.13 (executor 1) (117/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 32.0 in stage 6.0 (TID 236) in 765 ms on 10.60.24.13 (executor 1) (118/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 70.0 in stage 6.0 (TID 274) in 763 ms on 10.60.24.13 (executor 1) (119/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 46.0 in stage 6.0 (TID 250) in 764 ms on 10.60.24.13 (executor 1) (120/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 26.0 in stage 6.0 (TID 230) in 766 ms on 10.60.24.13 (executor 1) (121/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 136.0 in stage 6.0 (TID 340) in 760 ms on 10.60.24.13 (executor 1) (122/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 76.0 in stage 6.0 (TID 280) in 763 ms on 10.60.24.13 (executor 1) (123/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 106.0 in stage 6.0 (TID 310) in 761 ms on 10.60.24.13 (executor 1) (124/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 112.0 in stage 6.0 (TID 316) in 761 ms on 10.60.24.13 (executor 1) (125/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 80.0 in stage 6.0 (TID 284) in 763 ms on 10.60.24.13 (executor 1) (126/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 28.0 in stage 6.0 (TID 232) in 767 ms on 10.60.24.13 (executor 1) (127/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 108.0 in stage 6.0 (TID 312) in 762 ms on 10.60.24.13 (executor 1) (128/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 82.0 in stage 6.0 (TID 286) in 764 ms on 10.60.24.13 (executor 1) (129/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 182.0 in stage 6.0 (TID 386) in 758 ms on 10.60.24.13 (executor 1) (130/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 176.0 in stage 6.0 (TID 380) in 759 ms on 10.60.24.13 (executor 1) (131/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 64.0 in stage 6.0 (TID 268) in 767 ms on 10.60.24.13 (executor 1) (132/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 140.0 in stage 6.0 (TID 344) in 763 ms on 10.60.24.13 (executor 1) (133/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 20.0 in stage 6.0 (TID 224) in 770 ms on 10.60.24.13 (executor 1) (134/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 160.0 in stage 6.0 (TID 364) in 761 ms on 10.60.24.13 (executor 1) (135/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 184.0 in stage 6.0 (TID 388) in 763 ms on 10.60.24.13 (executor 1) (136/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 162.0 in stage 6.0 (TID 366) in 764 ms on 10.60.24.13 (executor 1) (137/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 92.0 in stage 6.0 (TID 296) in 770 ms on 10.60.24.13 (executor 1) (138/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 54.0 in stage 6.0 (TID 258) in 771 ms on 10.60.24.13 (executor 1) (139/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 38.0 in stage 6.0 (TID 242) in 773 ms on 10.60.24.13 (executor 1) (140/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 170.0 in stage 6.0 (TID 374) in 765 ms on 10.60.24.13 (executor 1) (141/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 4.0 in stage 6.0 (TID 208) in 776 ms on 10.60.24.13 (executor 1) (142/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 66.0 in stage 6.0 (TID 270) in 772 ms on 10.60.24.13 (executor 1) (143/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 72.0 in stage 6.0 (TID 276) in 773 ms on 10.60.24.13 (executor 1) (144/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 128.0 in stage 6.0 (TID 332) in 769 ms on 10.60.24.13 (executor 1) (145/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 24.0 in stage 6.0 (TID 228) in 777 ms on 10.60.24.13 (executor 1) (146/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 34.0 in stage 6.0 (TID 238) in 776 ms on 10.60.24.13 (executor 1) (147/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 90.0 in stage 6.0 (TID 294) in 773 ms on 10.60.24.13 (executor 1) (148/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 174.0 in stage 6.0 (TID 378) in 767 ms on 10.60.24.13 (executor 1) (149/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 110.0 in stage 6.0 (TID 314) in 771 ms on 10.60.24.13 (executor 1) (150/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 132.0 in stage 6.0 (TID 336) in 770 ms on 10.60.24.13 (executor 1) (151/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 88.0 in stage 6.0 (TID 292) in 773 ms on 10.60.24.13 (executor 1) (152/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 104.0 in stage 6.0 (TID 308) in 773 ms on 10.60.24.13 (executor 1) (153/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 190.0 in stage 6.0 (TID 394) in 767 ms on 10.60.24.13 (executor 1) (154/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 58.0 in stage 6.0 (TID 262) in 775 ms on 10.60.24.13 (executor 1) (155/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 168.0 in stage 6.0 (TID 372) in 768 ms on 10.60.24.13 (executor 1) (156/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 172.0 in stage 6.0 (TID 376) in 768 ms on 10.60.24.13 (executor 1) (157/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 186.0 in stage 6.0 (TID 390) in 767 ms on 10.60.24.13 (executor 1) (158/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 86.0 in stage 6.0 (TID 290) in 775 ms on 10.60.24.13 (executor 1) (159/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 42.0 in stage 6.0 (TID 246) in 777 ms on 10.60.24.13 (executor 1) (160/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 30.0 in stage 6.0 (TID 234) in 779 ms on 10.60.24.13 (executor 1) (161/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 12.0 in stage 6.0 (TID 216) in 780 ms on 10.60.24.13 (executor 1) (162/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 102.0 in stage 6.0 (TID 306) in 775 ms on 10.60.24.13 (executor 1) (163/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 56.0 in stage 6.0 (TID 260) in 777 ms on 10.60.24.13 (executor 1) (164/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 118.0 in stage 6.0 (TID 322) in 774 ms on 10.60.24.13 (executor 1) (165/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 52.0 in stage 6.0 (TID 256) in 778 ms on 10.60.24.13 (executor 1) (166/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 50.0 in stage 6.0 (TID 254) in 778 ms on 10.60.24.13 (executor 1) (167/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 8.0 in stage 6.0 (TID 212) in 781 ms on 10.60.24.13 (executor 1) (168/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 134.0 in stage 6.0 (TID 338) in 774 ms on 10.60.24.13 (executor 1) (169/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 68.0 in stage 6.0 (TID 272) in 778 ms on 10.60.24.13 (executor 1) (170/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 48.0 in stage 6.0 (TID 252) in 779 ms on 10.60.24.13 (executor 1) (171/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 122.0 in stage 6.0 (TID 326) in 775 ms on 10.60.24.13 (executor 1) (172/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 44.0 in stage 6.0 (TID 248) in 779 ms on 10.60.24.13 (executor 1) (173/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 16.0 in stage 6.0 (TID 220) in 781 ms on 10.60.24.13 (executor 1) (174/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 60.0 in stage 6.0 (TID 264) in 778 ms on 10.60.24.13 (executor 1) (175/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 98.0 in stage 6.0 (TID 302) in 776 ms on 10.60.24.13 (executor 1) (176/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 18.0 in stage 6.0 (TID 222) in 782 ms on 10.60.24.13 (executor 1) (177/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 148.0 in stage 6.0 (TID 352) in 774 ms on 10.60.24.13 (executor 1) (178/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 62.0 in stage 6.0 (TID 266) in 779 ms on 10.60.24.13 (executor 1) (179/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 130.0 in stage 6.0 (TID 334) in 775 ms on 10.60.24.13 (executor 1) (180/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 142.0 in stage 6.0 (TID 346) in 775 ms on 10.60.24.13 (executor 1) (181/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 94.0 in stage 6.0 (TID 298) in 777 ms on 10.60.24.13 (executor 1) (182/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 154.0 in stage 6.0 (TID 358) in 775 ms on 10.60.24.13 (executor 1) (183/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 84.0 in stage 6.0 (TID 288) in 779 ms on 10.60.24.13 (executor 1) (184/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 40.0 in stage 6.0 (TID 244) in 782 ms on 10.60.24.13 (executor 1) (185/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 206) in 784 ms on 10.60.24.13 (executor 1) (186/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 204) in 784 ms on 10.60.24.13 (executor 1) (187/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 144.0 in stage 6.0 (TID 348) in 775 ms on 10.60.24.13 (executor 1) (188/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 166.0 in stage 6.0 (TID 370) in 773 ms on 10.60.24.13 (executor 1) (189/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 138.0 in stage 6.0 (TID 342) in 776 ms on 10.60.24.13 (executor 1) (190/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 188.0 in stage 6.0 (TID 392) in 772 ms on 10.60.24.13 (executor 1) (191/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 126.0 in stage 6.0 (TID 330) in 776 ms on 10.60.24.13 (executor 1) (192/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 164.0 in stage 6.0 (TID 368) in 773 ms on 10.60.24.13 (executor 1) (193/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 100.0 in stage 6.0 (TID 304) in 778 ms on 10.60.24.13 (executor 1) (194/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 156.0 in stage 6.0 (TID 360) in 774 ms on 10.60.24.13 (executor 1) (195/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 120.0 in stage 6.0 (TID 324) in 778 ms on 10.60.24.13 (executor 1) (196/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 158.0 in stage 6.0 (TID 362) in 775 ms on 10.60.24.13 (executor 1) (197/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 36.0 in stage 6.0 (TID 240) in 783 ms on 10.60.24.13 (executor 1) (198/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 146.0 in stage 6.0 (TID 350) in 776 ms on 10.60.24.13 (executor 1) (199/200) 23/06/28 07:48:48 INFO TaskSetManager: Finished task 22.0 in stage 6.0 (TID 226) in 934 ms on 10.60.24.13 (executor 1) (200/200) 23/06/28 07:48:48 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:48 INFO DAGScheduler: ShuffleMapStage 6 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.967 s 23/06/28 07:48:48 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:48 INFO DAGScheduler: running: Set() 23/06/28 07:48:48 INFO DAGScheduler: waiting: Set(ShuffleMapStage 7) 23/06/28 07:48:48 INFO DAGScheduler: failed: Set() 23/06/28 07:48:48 INFO DAGScheduler: Submitting ShuffleMapStage 7 (MapPartitionsRDD[44] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:48 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:48 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:48 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:48 INFO DAGScheduler: Submitting 200 missing tasks from ShuffleMapStage 7 (MapPartitionsRDD[44] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:48 INFO TaskSchedulerImpl: Adding task set 7.0 with 200 tasks resource profile 0 23/06/28 07:48:48 INFO FairSchedulableBuilder: Added task set TaskSet_7.0 tasks to pool 7670323585580043917 23/06/28 07:48:48 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 404) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 1.0 in stage 7.0 (TID 405) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 2.0 in stage 7.0 (TID 406) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 3.0 in stage 7.0 (TID 407) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 4.0 in stage 7.0 (TID 408) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 5.0 in stage 7.0 (TID 409) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 6.0 in stage 7.0 (TID 410) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 7.0 in stage 7.0 (TID 411) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 8.0 in stage 7.0 (TID 412) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 9.0 in stage 7.0 (TID 413) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 10.0 in stage 7.0 (TID 414) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 11.0 in stage 7.0 (TID 415) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 12.0 in stage 7.0 (TID 416) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 13.0 in stage 7.0 (TID 417) (10.60.24.10, executor 0, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 14.0 in stage 7.0 (TID 418) (10.60.24.13, executor 1, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 15.0 in stage 7.0 (TID 419) (10.60.24.10, executor 0, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 16.0 in stage 7.0 (TID 420) (10.60.24.13, executor 1, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 17.0 in stage 7.0 (TID 421) (10.60.24.10, executor 0, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 18.0 in stage 7.0 (TID 422) (10.60.24.13, executor 1, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 19.0 in stage 7.0 (TID 423) (10.60.24.10, executor 0, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 20.0 in stage 7.0 (TID 424) (10.60.24.13, executor 1, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 21.0 in stage 7.0 (TID 425) (10.60.24.10, executor 0, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 22.0 in stage 7.0 (TID 426) (10.60.24.13, executor 1, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 23.0 in stage 7.0 (TID 427) (10.60.24.10, executor 0, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 24.0 in stage 7.0 (TID 428) (10.60.24.13, executor 1, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 25.0 in stage 7.0 (TID 429) (10.60.24.10, executor 0, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 26.0 in stage 7.0 (TID 430) (10.60.24.13, executor 1, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 27.0 in stage 7.0 (TID 431) (10.60.24.10, executor 0, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 28.0 in stage 7.0 (TID 432) (10.60.24.13, executor 1, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 29.0 in stage 7.0 (TID 433) (10.60.24.10, executor 0, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 30.0 in stage 7.0 (TID 434) (10.60.24.13, executor 1, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 31.0 in stage 7.0 (TID 435) (10.60.24.10, executor 0, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 32.0 in stage 7.0 (TID 436) (10.60.24.13, executor 1, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 33.0 in stage 7.0 (TID 437) (10.60.24.10, executor 0, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 34.0 in stage 7.0 (TID 438) (10.60.24.13, executor 1, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 35.0 in stage 7.0 (TID 439) (10.60.24.10, executor 0, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 36.0 in stage 7.0 (TID 440) (10.60.24.13, executor 1, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 37.0 in stage 7.0 (TID 441) (10.60.24.10, executor 0, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 38.0 in stage 7.0 (TID 442) (10.60.24.13, executor 1, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 39.0 in stage 7.0 (TID 443) (10.60.24.10, executor 0, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 40.0 in stage 7.0 (TID 444) (10.60.24.13, executor 1, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 41.0 in stage 7.0 (TID 445) (10.60.24.10, executor 0, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 42.0 in stage 7.0 (TID 446) (10.60.24.13, executor 1, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 43.0 in stage 7.0 (TID 447) (10.60.24.10, executor 0, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 44.0 in stage 7.0 (TID 448) (10.60.24.13, executor 1, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 45.0 in stage 7.0 (TID 449) (10.60.24.10, executor 0, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 46.0 in stage 7.0 (TID 450) (10.60.24.13, executor 1, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 47.0 in stage 7.0 (TID 451) (10.60.24.10, executor 0, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 48.0 in stage 7.0 (TID 452) (10.60.24.13, executor 1, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 49.0 in stage 7.0 (TID 453) (10.60.24.10, executor 0, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 50.0 in stage 7.0 (TID 454) (10.60.24.13, executor 1, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 51.0 in stage 7.0 (TID 455) (10.60.24.10, executor 0, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 52.0 in stage 7.0 (TID 456) (10.60.24.13, executor 1, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 53.0 in stage 7.0 (TID 457) (10.60.24.10, executor 0, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 54.0 in stage 7.0 (TID 458) (10.60.24.13, executor 1, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 55.0 in stage 7.0 (TID 459) (10.60.24.10, executor 0, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 56.0 in stage 7.0 (TID 460) (10.60.24.13, executor 1, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 57.0 in stage 7.0 (TID 461) (10.60.24.10, executor 0, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 58.0 in stage 7.0 (TID 462) (10.60.24.13, executor 1, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 59.0 in stage 7.0 (TID 463) (10.60.24.10, executor 0, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 60.0 in stage 7.0 (TID 464) (10.60.24.13, executor 1, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 61.0 in stage 7.0 (TID 465) (10.60.24.10, executor 0, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 62.0 in stage 7.0 (TID 466) (10.60.24.13, executor 1, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 63.0 in stage 7.0 (TID 467) (10.60.24.10, executor 0, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 64.0 in stage 7.0 (TID 468) (10.60.24.13, executor 1, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 65.0 in stage 7.0 (TID 469) (10.60.24.10, executor 0, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 66.0 in stage 7.0 (TID 470) (10.60.24.13, executor 1, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 67.0 in stage 7.0 (TID 471) (10.60.24.10, executor 0, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 68.0 in stage 7.0 (TID 472) (10.60.24.13, executor 1, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 69.0 in stage 7.0 (TID 473) (10.60.24.10, executor 0, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 70.0 in stage 7.0 (TID 474) (10.60.24.13, executor 1, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 71.0 in stage 7.0 (TID 475) (10.60.24.10, executor 0, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 72.0 in stage 7.0 (TID 476) (10.60.24.13, executor 1, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 73.0 in stage 7.0 (TID 477) (10.60.24.10, executor 0, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 74.0 in stage 7.0 (TID 478) (10.60.24.13, executor 1, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 75.0 in stage 7.0 (TID 479) (10.60.24.10, executor 0, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 76.0 in stage 7.0 (TID 480) (10.60.24.13, executor 1, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 77.0 in stage 7.0 (TID 481) (10.60.24.10, executor 0, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 78.0 in stage 7.0 (TID 482) (10.60.24.13, executor 1, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 79.0 in stage 7.0 (TID 483) (10.60.24.10, executor 0, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 80.0 in stage 7.0 (TID 484) (10.60.24.13, executor 1, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 81.0 in stage 7.0 (TID 485) (10.60.24.10, executor 0, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 82.0 in stage 7.0 (TID 486) (10.60.24.13, executor 1, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 83.0 in stage 7.0 (TID 487) (10.60.24.10, executor 0, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 84.0 in stage 7.0 (TID 488) (10.60.24.13, executor 1, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 85.0 in stage 7.0 (TID 489) (10.60.24.10, executor 0, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 86.0 in stage 7.0 (TID 490) (10.60.24.13, executor 1, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 87.0 in stage 7.0 (TID 491) (10.60.24.10, executor 0, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 88.0 in stage 7.0 (TID 492) (10.60.24.13, executor 1, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 89.0 in stage 7.0 (TID 493) (10.60.24.10, executor 0, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 90.0 in stage 7.0 (TID 494) (10.60.24.13, executor 1, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 91.0 in stage 7.0 (TID 495) (10.60.24.10, executor 0, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 92.0 in stage 7.0 (TID 496) (10.60.24.13, executor 1, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 93.0 in stage 7.0 (TID 497) (10.60.24.10, executor 0, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 94.0 in stage 7.0 (TID 498) (10.60.24.13, executor 1, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 95.0 in stage 7.0 (TID 499) (10.60.24.10, executor 0, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 96.0 in stage 7.0 (TID 500) (10.60.24.13, executor 1, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 97.0 in stage 7.0 (TID 501) (10.60.24.10, executor 0, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 98.0 in stage 7.0 (TID 502) (10.60.24.13, executor 1, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 99.0 in stage 7.0 (TID 503) (10.60.24.10, executor 0, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 100.0 in stage 7.0 (TID 504) (10.60.24.13, executor 1, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 101.0 in stage 7.0 (TID 505) (10.60.24.10, executor 0, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 102.0 in stage 7.0 (TID 506) (10.60.24.13, executor 1, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 103.0 in stage 7.0 (TID 507) (10.60.24.10, executor 0, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 104.0 in stage 7.0 (TID 508) (10.60.24.13, executor 1, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 105.0 in stage 7.0 (TID 509) (10.60.24.10, executor 0, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 106.0 in stage 7.0 (TID 510) (10.60.24.13, executor 1, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 107.0 in stage 7.0 (TID 511) (10.60.24.10, executor 0, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 108.0 in stage 7.0 (TID 512) (10.60.24.13, executor 1, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 109.0 in stage 7.0 (TID 513) (10.60.24.10, executor 0, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 110.0 in stage 7.0 (TID 514) (10.60.24.13, executor 1, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 111.0 in stage 7.0 (TID 515) (10.60.24.10, executor 0, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 112.0 in stage 7.0 (TID 516) (10.60.24.13, executor 1, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 113.0 in stage 7.0 (TID 517) (10.60.24.10, executor 0, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 114.0 in stage 7.0 (TID 518) (10.60.24.13, executor 1, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 115.0 in stage 7.0 (TID 519) (10.60.24.10, executor 0, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 116.0 in stage 7.0 (TID 520) (10.60.24.13, executor 1, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 117.0 in stage 7.0 (TID 521) (10.60.24.10, executor 0, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 118.0 in stage 7.0 (TID 522) (10.60.24.13, executor 1, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 119.0 in stage 7.0 (TID 523) (10.60.24.10, executor 0, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 120.0 in stage 7.0 (TID 524) (10.60.24.13, executor 1, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 121.0 in stage 7.0 (TID 525) (10.60.24.10, executor 0, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 122.0 in stage 7.0 (TID 526) (10.60.24.13, executor 1, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 123.0 in stage 7.0 (TID 527) (10.60.24.10, executor 0, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 124.0 in stage 7.0 (TID 528) (10.60.24.13, executor 1, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 125.0 in stage 7.0 (TID 529) (10.60.24.10, executor 0, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 126.0 in stage 7.0 (TID 530) (10.60.24.13, executor 1, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 127.0 in stage 7.0 (TID 531) (10.60.24.10, executor 0, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 128.0 in stage 7.0 (TID 532) (10.60.24.13, executor 1, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 129.0 in stage 7.0 (TID 533) (10.60.24.10, executor 0, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 130.0 in stage 7.0 (TID 534) (10.60.24.13, executor 1, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 131.0 in stage 7.0 (TID 535) (10.60.24.10, executor 0, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 132.0 in stage 7.0 (TID 536) (10.60.24.13, executor 1, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 133.0 in stage 7.0 (TID 537) (10.60.24.10, executor 0, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 134.0 in stage 7.0 (TID 538) (10.60.24.13, executor 1, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 135.0 in stage 7.0 (TID 539) (10.60.24.10, executor 0, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 136.0 in stage 7.0 (TID 540) (10.60.24.13, executor 1, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 137.0 in stage 7.0 (TID 541) (10.60.24.10, executor 0, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 138.0 in stage 7.0 (TID 542) (10.60.24.13, executor 1, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 139.0 in stage 7.0 (TID 543) (10.60.24.10, executor 0, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 140.0 in stage 7.0 (TID 544) (10.60.24.13, executor 1, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 141.0 in stage 7.0 (TID 545) (10.60.24.10, executor 0, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 142.0 in stage 7.0 (TID 546) (10.60.24.13, executor 1, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 143.0 in stage 7.0 (TID 547) (10.60.24.10, executor 0, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 144.0 in stage 7.0 (TID 548) (10.60.24.13, executor 1, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 145.0 in stage 7.0 (TID 549) (10.60.24.10, executor 0, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 146.0 in stage 7.0 (TID 550) (10.60.24.13, executor 1, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 147.0 in stage 7.0 (TID 551) (10.60.24.10, executor 0, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 148.0 in stage 7.0 (TID 552) (10.60.24.13, executor 1, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 149.0 in stage 7.0 (TID 553) (10.60.24.10, executor 0, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 150.0 in stage 7.0 (TID 554) (10.60.24.13, executor 1, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 151.0 in stage 7.0 (TID 555) (10.60.24.10, executor 0, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 152.0 in stage 7.0 (TID 556) (10.60.24.13, executor 1, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 153.0 in stage 7.0 (TID 557) (10.60.24.10, executor 0, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 154.0 in stage 7.0 (TID 558) (10.60.24.13, executor 1, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 155.0 in stage 7.0 (TID 559) (10.60.24.10, executor 0, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 156.0 in stage 7.0 (TID 560) (10.60.24.13, executor 1, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 157.0 in stage 7.0 (TID 561) (10.60.24.10, executor 0, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 158.0 in stage 7.0 (TID 562) (10.60.24.13, executor 1, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 159.0 in stage 7.0 (TID 563) (10.60.24.10, executor 0, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 160.0 in stage 7.0 (TID 564) (10.60.24.13, executor 1, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 161.0 in stage 7.0 (TID 565) (10.60.24.10, executor 0, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 162.0 in stage 7.0 (TID 566) (10.60.24.13, executor 1, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 163.0 in stage 7.0 (TID 567) (10.60.24.10, executor 0, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 164.0 in stage 7.0 (TID 568) (10.60.24.13, executor 1, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 165.0 in stage 7.0 (TID 569) (10.60.24.10, executor 0, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 166.0 in stage 7.0 (TID 570) (10.60.24.13, executor 1, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 167.0 in stage 7.0 (TID 571) (10.60.24.10, executor 0, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 168.0 in stage 7.0 (TID 572) (10.60.24.13, executor 1, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 169.0 in stage 7.0 (TID 573) (10.60.24.10, executor 0, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 170.0 in stage 7.0 (TID 574) (10.60.24.13, executor 1, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 171.0 in stage 7.0 (TID 575) (10.60.24.10, executor 0, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 172.0 in stage 7.0 (TID 576) (10.60.24.13, executor 1, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 173.0 in stage 7.0 (TID 577) (10.60.24.10, executor 0, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 174.0 in stage 7.0 (TID 578) (10.60.24.13, executor 1, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 175.0 in stage 7.0 (TID 579) (10.60.24.10, executor 0, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 176.0 in stage 7.0 (TID 580) (10.60.24.13, executor 1, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 177.0 in stage 7.0 (TID 581) (10.60.24.10, executor 0, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 178.0 in stage 7.0 (TID 582) (10.60.24.13, executor 1, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 179.0 in stage 7.0 (TID 583) (10.60.24.10, executor 0, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 180.0 in stage 7.0 (TID 584) (10.60.24.13, executor 1, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 181.0 in stage 7.0 (TID 585) (10.60.24.10, executor 0, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 182.0 in stage 7.0 (TID 586) (10.60.24.13, executor 1, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 183.0 in stage 7.0 (TID 587) (10.60.24.10, executor 0, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 184.0 in stage 7.0 (TID 588) (10.60.24.13, executor 1, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 185.0 in stage 7.0 (TID 589) (10.60.24.10, executor 0, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 186.0 in stage 7.0 (TID 590) (10.60.24.13, executor 1, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 187.0 in stage 7.0 (TID 591) (10.60.24.10, executor 0, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 188.0 in stage 7.0 (TID 592) (10.60.24.13, executor 1, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 189.0 in stage 7.0 (TID 593) (10.60.24.10, executor 0, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 190.0 in stage 7.0 (TID 594) (10.60.24.13, executor 1, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO TaskSetManager: Starting task 191.0 in stage 7.0 (TID 595) (10.60.24.10, executor 0, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:48 INFO MemoryStore: Block broadcast_15 stored as values in memory (estimated size 165.7 KiB, free 750.7 GiB) 23/06/28 07:48:48 INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 37.9 KiB, free 750.7 GiB) 23/06/28 07:48:48 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on 10.60.24.15:40507 (size: 37.9 KiB, free: 750.7 GiB) 23/06/28 07:48:48 INFO SparkContext: Created broadcast 15 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:48 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on 10.60.24.13:44035 (size: 37.9 KiB, free: 487.7 GiB) 23/06/28 07:48:48 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on 10.60.24.10:41041 (size: 37.9 KiB, free: 487.7 GiB) 23/06/28 07:48:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 10.60.24.10:44288 23/06/28 07:48:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 10.60.24.13:58548 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on 10.60.24.10:41041 (size: 2.2 KiB, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_57 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_117 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_91 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_77 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_87 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_51 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_159 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_15 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_113 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_157 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_83 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_105 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_129 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_5 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_147 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_1 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_137 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_69 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_143 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_145 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_151 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_23 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_97 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_43 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_133 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_99 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_21 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_25 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_177 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_85 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_81 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_75 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_39 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_111 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_123 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_119 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_59 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_171 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_17 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_163 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_167 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_121 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_65 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_149 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_155 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_19 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_63 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_9 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_47 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_55 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_191 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_153 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_141 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_181 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_131 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_183 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_31 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_71 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_173 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_7 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_27 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_79 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_175 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_187 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_37 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_185 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_135 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_189 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_165 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_53 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_139 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_67 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_127 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_103 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_41 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_61 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_29 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_125 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_93 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_3 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_161 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on 10.60.24.13:44035 (size: 2.2 KiB, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_140 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_162 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_90 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_32 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_0 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_104 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_62 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_4 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_66 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_186 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_84 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_124 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_130 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_154 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_108 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_96 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_58 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_60 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_92 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_118 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_22 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_150 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_168 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_6 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_98 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_152 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_88 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_56 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_148 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_74 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_116 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_94 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_50 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_46 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_142 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_190 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_78 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_34 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_138 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_52 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_54 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_166 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_18 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_82 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_44 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_132 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_134 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_178 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_188 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_16 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_102 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_136 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_110 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_146 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_172 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_38 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_156 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_158 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_176 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_126 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_72 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_14 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_40 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_170 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_122 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_114 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_80 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_180 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_164 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_64 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_76 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_160 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_86 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_36 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_30 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_24 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_112 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_106 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_174 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_120 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_42 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_182 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_20 in memory on 10.60.24.13:44035 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 192.0 in stage 7.0 (TID 596) (10.60.24.10, executor 0, partition 192, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 15.0 in stage 7.0 (TID 419) in 648 ms on 10.60.24.10 (executor 0) (1/200) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 193.0 in stage 7.0 (TID 597) (10.60.24.10, executor 0, partition 193, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 194.0 in stage 7.0 (TID 598) (10.60.24.10, executor 0, partition 194, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 21.0 in stage 7.0 (TID 425) in 650 ms on 10.60.24.10 (executor 0) (2/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 97.0 in stage 7.0 (TID 501) in 646 ms on 10.60.24.10 (executor 0) (3/200) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 195.0 in stage 7.0 (TID 599) (10.60.24.10, executor 0, partition 195, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 196.0 in stage 7.0 (TID 600) (10.60.24.10, executor 0, partition 196, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 197.0 in stage 7.0 (TID 601) (10.60.24.10, executor 0, partition 197, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 198.0 in stage 7.0 (TID 602) (10.60.24.10, executor 0, partition 198, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 113.0 in stage 7.0 (TID 517) in 642 ms on 10.60.24.10 (executor 0) (4/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 189.0 in stage 7.0 (TID 593) in 639 ms on 10.60.24.10 (executor 0) (5/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 37.0 in stage 7.0 (TID 441) in 649 ms on 10.60.24.10 (executor 0) (6/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 145.0 in stage 7.0 (TID 549) in 641 ms on 10.60.24.10 (executor 0) (7/200) 23/06/28 07:48:49 INFO TaskSetManager: Starting task 199.0 in stage 7.0 (TID 603) (10.60.24.10, executor 0, partition 199, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 1.0 in stage 7.0 (TID 405) in 652 ms on 10.60.24.10 (executor 0) (8/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_107 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 19.0 in stage 7.0 (TID 423) in 651 ms on 10.60.24.10 (executor 0) (9/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 129.0 in stage 7.0 (TID 533) in 642 ms on 10.60.24.10 (executor 0) (10/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_169 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 67.0 in stage 7.0 (TID 471) in 649 ms on 10.60.24.10 (executor 0) (11/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_109 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 125.0 in stage 7.0 (TID 529) in 643 ms on 10.60.24.10 (executor 0) (12/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_89 in memory on 10.60.24.10:41041 (size: 1384.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 3.0 in stage 7.0 (TID 407) in 652 ms on 10.60.24.10 (executor 0) (13/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 81.0 in stage 7.0 (TID 485) in 649 ms on 10.60.24.10 (executor 0) (14/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_101 in memory on 10.60.24.10:41041 (size: 1048.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 85.0 in stage 7.0 (TID 489) in 649 ms on 10.60.24.10 (executor 0) (15/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 5.0 in stage 7.0 (TID 409) in 653 ms on 10.60.24.10 (executor 0) (16/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_49 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 161.0 in stage 7.0 (TID 565) in 642 ms on 10.60.24.10 (executor 0) (17/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_11 in memory on 10.60.24.10:41041 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 123.0 in stage 7.0 (TID 527) in 644 ms on 10.60.24.10 (executor 0) (18/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_73 in memory on 10.60.24.10:41041 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 79.0 in stage 7.0 (TID 483) in 650 ms on 10.60.24.10 (executor 0) (19/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 177.0 in stage 7.0 (TID 581) in 642 ms on 10.60.24.10 (executor 0) (20/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_45 in memory on 10.60.24.10:41041 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 191.0 in stage 7.0 (TID 595) in 641 ms on 10.60.24.10 (executor 0) (21/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 9.0 in stage 7.0 (TID 413) in 652 ms on 10.60.24.10 (executor 0) (22/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_13 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_179 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 133.0 in stage 7.0 (TID 537) in 643 ms on 10.60.24.10 (executor 0) (23/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 41.0 in stage 7.0 (TID 445) in 651 ms on 10.60.24.10 (executor 0) (24/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 171.0 in stage 7.0 (TID 575) in 642 ms on 10.60.24.10 (executor 0) (25/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_35 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 187.0 in stage 7.0 (TID 591) in 643 ms on 10.60.24.10 (executor 0) (26/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 65.0 in stage 7.0 (TID 469) in 651 ms on 10.60.24.10 (executor 0) (27/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_95 in memory on 10.60.24.10:41041 (size: 1488.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_115 in memory on 10.60.24.10:41041 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 63.0 in stage 7.0 (TID 467) in 651 ms on 10.60.24.10 (executor 0) (28/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 77.0 in stage 7.0 (TID 481) in 651 ms on 10.60.24.10 (executor 0) (29/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 165.0 in stage 7.0 (TID 569) in 644 ms on 10.60.24.10 (executor 0) (30/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 31.0 in stage 7.0 (TID 435) in 653 ms on 10.60.24.10 (executor 0) (31/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 153.0 in stage 7.0 (TID 557) in 646 ms on 10.60.24.10 (executor 0) (32/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 157.0 in stage 7.0 (TID 561) in 645 ms on 10.60.24.10 (executor 0) (33/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_33 in memory on 10.60.24.10:41041 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 139.0 in stage 7.0 (TID 543) in 646 ms on 10.60.24.10 (executor 0) (34/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 105.0 in stage 7.0 (TID 509) in 650 ms on 10.60.24.10 (executor 0) (35/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 181.0 in stage 7.0 (TID 585) in 645 ms on 10.60.24.10 (executor 0) (36/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 175.0 in stage 7.0 (TID 579) in 645 ms on 10.60.24.10 (executor 0) (37/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 47.0 in stage 7.0 (TID 451) in 654 ms on 10.60.24.10 (executor 0) (38/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 87.0 in stage 7.0 (TID 491) in 653 ms on 10.60.24.10 (executor 0) (39/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 147.0 in stage 7.0 (TID 551) in 647 ms on 10.60.24.10 (executor 0) (40/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 51.0 in stage 7.0 (TID 455) in 655 ms on 10.60.24.10 (executor 0) (41/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 17.0 in stage 7.0 (TID 421) in 656 ms on 10.60.24.10 (executor 0) (42/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 111.0 in stage 7.0 (TID 515) in 649 ms on 10.60.24.10 (executor 0) (43/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 159.0 in stage 7.0 (TID 563) in 647 ms on 10.60.24.10 (executor 0) (44/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 27.0 in stage 7.0 (TID 431) in 657 ms on 10.60.24.10 (executor 0) (45/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 141.0 in stage 7.0 (TID 545) in 649 ms on 10.60.24.10 (executor 0) (46/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 23.0 in stage 7.0 (TID 427) in 658 ms on 10.60.24.10 (executor 0) (47/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 61.0 in stage 7.0 (TID 465) in 656 ms on 10.60.24.10 (executor 0) (48/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 69.0 in stage 7.0 (TID 473) in 657 ms on 10.60.24.10 (executor 0) (49/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 137.0 in stage 7.0 (TID 541) in 650 ms on 10.60.24.10 (executor 0) (50/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 83.0 in stage 7.0 (TID 487) in 656 ms on 10.60.24.10 (executor 0) (51/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 135.0 in stage 7.0 (TID 539) in 650 ms on 10.60.24.10 (executor 0) (52/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 149.0 in stage 7.0 (TID 553) in 650 ms on 10.60.24.10 (executor 0) (53/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 155.0 in stage 7.0 (TID 559) in 650 ms on 10.60.24.10 (executor 0) (54/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 183.0 in stage 7.0 (TID 587) in 650 ms on 10.60.24.10 (executor 0) (55/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 39.0 in stage 7.0 (TID 443) in 659 ms on 10.60.24.10 (executor 0) (56/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 131.0 in stage 7.0 (TID 535) in 651 ms on 10.60.24.10 (executor 0) (57/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 119.0 in stage 7.0 (TID 523) in 652 ms on 10.60.24.10 (executor 0) (58/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 167.0 in stage 7.0 (TID 571) in 651 ms on 10.60.24.10 (executor 0) (59/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 127.0 in stage 7.0 (TID 531) in 652 ms on 10.60.24.10 (executor 0) (60/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 75.0 in stage 7.0 (TID 479) in 659 ms on 10.60.24.10 (executor 0) (61/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 57.0 in stage 7.0 (TID 461) in 661 ms on 10.60.24.10 (executor 0) (62/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 121.0 in stage 7.0 (TID 525) in 654 ms on 10.60.24.10 (executor 0) (63/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 173.0 in stage 7.0 (TID 577) in 652 ms on 10.60.24.10 (executor 0) (64/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 103.0 in stage 7.0 (TID 507) in 657 ms on 10.60.24.10 (executor 0) (65/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 71.0 in stage 7.0 (TID 475) in 660 ms on 10.60.24.10 (executor 0) (66/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 55.0 in stage 7.0 (TID 459) in 662 ms on 10.60.24.10 (executor 0) (67/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 99.0 in stage 7.0 (TID 503) in 659 ms on 10.60.24.10 (executor 0) (68/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 93.0 in stage 7.0 (TID 497) in 659 ms on 10.60.24.10 (executor 0) (69/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 59.0 in stage 7.0 (TID 463) in 661 ms on 10.60.24.10 (executor 0) (70/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 151.0 in stage 7.0 (TID 555) in 654 ms on 10.60.24.10 (executor 0) (71/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 7.0 in stage 7.0 (TID 411) in 664 ms on 10.60.24.10 (executor 0) (72/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 53.0 in stage 7.0 (TID 457) in 663 ms on 10.60.24.10 (executor 0) (73/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 25.0 in stage 7.0 (TID 429) in 664 ms on 10.60.24.10 (executor 0) (74/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 143.0 in stage 7.0 (TID 547) in 656 ms on 10.60.24.10 (executor 0) (75/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 43.0 in stage 7.0 (TID 447) in 664 ms on 10.60.24.10 (executor 0) (76/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 185.0 in stage 7.0 (TID 589) in 655 ms on 10.60.24.10 (executor 0) (77/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 117.0 in stage 7.0 (TID 521) in 658 ms on 10.60.24.10 (executor 0) (78/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 91.0 in stage 7.0 (TID 495) in 663 ms on 10.60.24.10 (executor 0) (79/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 29.0 in stage 7.0 (TID 433) in 666 ms on 10.60.24.10 (executor 0) (80/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 163.0 in stage 7.0 (TID 567) in 657 ms on 10.60.24.10 (executor 0) (81/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 35.0 in stage 7.0 (TID 439) in 667 ms on 10.60.24.10 (executor 0) (82/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 101.0 in stage 7.0 (TID 505) in 664 ms on 10.60.24.10 (executor 0) (83/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_199 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_192 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 95.0 in stage 7.0 (TID 499) in 665 ms on 10.60.24.10 (executor 0) (84/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 73.0 in stage 7.0 (TID 477) in 667 ms on 10.60.24.10 (executor 0) (85/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_194 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_195 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_197 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 13.0 in stage 7.0 (TID 417) in 669 ms on 10.60.24.10 (executor 0) (86/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_193 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_198 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_196 in memory on 10.60.24.10:41041 (size: 24.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 49.0 in stage 7.0 (TID 453) in 669 ms on 10.60.24.10 (executor 0) (87/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 169.0 in stage 7.0 (TID 573) in 661 ms on 10.60.24.10 (executor 0) (88/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 45.0 in stage 7.0 (TID 449) in 670 ms on 10.60.24.10 (executor 0) (89/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 179.0 in stage 7.0 (TID 583) in 661 ms on 10.60.24.10 (executor 0) (90/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 89.0 in stage 7.0 (TID 493) in 668 ms on 10.60.24.10 (executor 0) (91/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 115.0 in stage 7.0 (TID 519) in 664 ms on 10.60.24.10 (executor 0) (92/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 107.0 in stage 7.0 (TID 511) in 665 ms on 10.60.24.10 (executor 0) (93/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 109.0 in stage 7.0 (TID 513) in 664 ms on 10.60.24.10 (executor 0) (94/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 33.0 in stage 7.0 (TID 437) in 671 ms on 10.60.24.10 (executor 0) (95/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 199.0 in stage 7.0 (TID 603) in 22 ms on 10.60.24.10 (executor 0) (96/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 198.0 in stage 7.0 (TID 602) in 23 ms on 10.60.24.10 (executor 0) (97/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 11.0 in stage 7.0 (TID 415) in 673 ms on 10.60.24.10 (executor 0) (98/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 194.0 in stage 7.0 (TID 598) in 24 ms on 10.60.24.10 (executor 0) (99/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 192.0 in stage 7.0 (TID 596) in 26 ms on 10.60.24.10 (executor 0) (100/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 196.0 in stage 7.0 (TID 600) in 24 ms on 10.60.24.10 (executor 0) (101/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 197.0 in stage 7.0 (TID 601) in 24 ms on 10.60.24.10 (executor 0) (102/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 195.0 in stage 7.0 (TID 599) in 25 ms on 10.60.24.10 (executor 0) (103/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 193.0 in stage 7.0 (TID 597) in 26 ms on 10.60.24.10 (executor 0) (104/200) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_184 in memory on 10.60.24.13:44035 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_2 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_48 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_28 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_12 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_26 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_8 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_70 in memory on 10.60.24.13:44035 (size: 1072.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_68 in memory on 10.60.24.13:44035 (size: 1096.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_128 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_100 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_144 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added rdd_39_10 in memory on 10.60.24.13:44035 (size: 1320.0 B, free: 487.7 GiB) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 22.0 in stage 7.0 (TID 426) in 720 ms on 10.60.24.13 (executor 1) (105/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 98.0 in stage 7.0 (TID 502) in 717 ms on 10.60.24.13 (executor 1) (106/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 142.0 in stage 7.0 (TID 546) in 712 ms on 10.60.24.13 (executor 1) (107/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 64.0 in stage 7.0 (TID 468) in 719 ms on 10.60.24.13 (executor 1) (108/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 104.0 in stage 7.0 (TID 508) in 716 ms on 10.60.24.13 (executor 1) (109/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 40.0 in stage 7.0 (TID 444) in 720 ms on 10.60.24.13 (executor 1) (110/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 174.0 in stage 7.0 (TID 578) in 711 ms on 10.60.24.13 (executor 1) (111/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 148.0 in stage 7.0 (TID 552) in 713 ms on 10.60.24.13 (executor 1) (112/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 20.0 in stage 7.0 (TID 424) in 722 ms on 10.60.24.13 (executor 1) (113/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 168.0 in stage 7.0 (TID 572) in 712 ms on 10.60.24.13 (executor 1) (114/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 6.0 in stage 7.0 (TID 410) in 722 ms on 10.60.24.13 (executor 1) (115/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 8.0 in stage 7.0 (TID 412) in 723 ms on 10.60.24.13 (executor 1) (116/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 186.0 in stage 7.0 (TID 590) in 713 ms on 10.60.24.13 (executor 1) (117/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 58.0 in stage 7.0 (TID 462) in 722 ms on 10.60.24.13 (executor 1) (118/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 140.0 in stage 7.0 (TID 544) in 714 ms on 10.60.24.13 (executor 1) (119/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 82.0 in stage 7.0 (TID 486) in 720 ms on 10.60.24.13 (executor 1) (120/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 16.0 in stage 7.0 (TID 420) in 723 ms on 10.60.24.13 (executor 1) (121/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 94.0 in stage 7.0 (TID 498) in 720 ms on 10.60.24.13 (executor 1) (122/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 154.0 in stage 7.0 (TID 558) in 715 ms on 10.60.24.13 (executor 1) (123/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 404) in 725 ms on 10.60.24.13 (executor 1) (124/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 190.0 in stage 7.0 (TID 594) in 713 ms on 10.60.24.13 (executor 1) (125/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 88.0 in stage 7.0 (TID 492) in 721 ms on 10.60.24.13 (executor 1) (126/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 126.0 in stage 7.0 (TID 530) in 716 ms on 10.60.24.13 (executor 1) (127/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 122.0 in stage 7.0 (TID 526) in 717 ms on 10.60.24.13 (executor 1) (128/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 86.0 in stage 7.0 (TID 490) in 722 ms on 10.60.24.13 (executor 1) (129/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 176.0 in stage 7.0 (TID 580) in 715 ms on 10.60.24.13 (executor 1) (130/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 102.0 in stage 7.0 (TID 506) in 720 ms on 10.60.24.13 (executor 1) (131/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 66.0 in stage 7.0 (TID 470) in 723 ms on 10.60.24.13 (executor 1) (132/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 116.0 in stage 7.0 (TID 520) in 718 ms on 10.60.24.13 (executor 1) (133/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 162.0 in stage 7.0 (TID 566) in 716 ms on 10.60.24.13 (executor 1) (134/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 128.0 in stage 7.0 (TID 532) in 717 ms on 10.60.24.13 (executor 1) (135/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 100.0 in stage 7.0 (TID 504) in 721 ms on 10.60.24.13 (executor 1) (136/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 12.0 in stage 7.0 (TID 416) in 726 ms on 10.60.24.13 (executor 1) (137/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 2.0 in stage 7.0 (TID 406) in 728 ms on 10.60.24.13 (executor 1) (138/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 144.0 in stage 7.0 (TID 548) in 718 ms on 10.60.24.13 (executor 1) (139/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 70.0 in stage 7.0 (TID 474) in 725 ms on 10.60.24.13 (executor 1) (140/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 48.0 in stage 7.0 (TID 452) in 726 ms on 10.60.24.13 (executor 1) (141/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 28.0 in stage 7.0 (TID 432) in 727 ms on 10.60.24.13 (executor 1) (142/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 68.0 in stage 7.0 (TID 472) in 726 ms on 10.60.24.13 (executor 1) (143/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 10.0 in stage 7.0 (TID 414) in 728 ms on 10.60.24.13 (executor 1) (144/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 184.0 in stage 7.0 (TID 588) in 718 ms on 10.60.24.13 (executor 1) (145/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 26.0 in stage 7.0 (TID 430) in 728 ms on 10.60.24.13 (executor 1) (146/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 188.0 in stage 7.0 (TID 592) in 718 ms on 10.60.24.13 (executor 1) (147/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 182.0 in stage 7.0 (TID 586) in 719 ms on 10.60.24.13 (executor 1) (148/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 18.0 in stage 7.0 (TID 422) in 729 ms on 10.60.24.13 (executor 1) (149/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 164.0 in stage 7.0 (TID 568) in 719 ms on 10.60.24.13 (executor 1) (150/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 178.0 in stage 7.0 (TID 582) in 719 ms on 10.60.24.13 (executor 1) (151/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 76.0 in stage 7.0 (TID 480) in 728 ms on 10.60.24.13 (executor 1) (152/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 132.0 in stage 7.0 (TID 536) in 721 ms on 10.60.24.13 (executor 1) (153/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 4.0 in stage 7.0 (TID 408) in 731 ms on 10.60.24.13 (executor 1) (154/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 92.0 in stage 7.0 (TID 496) in 726 ms on 10.60.24.13 (executor 1) (155/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 72.0 in stage 7.0 (TID 476) in 728 ms on 10.60.24.13 (executor 1) (156/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 170.0 in stage 7.0 (TID 574) in 720 ms on 10.60.24.13 (executor 1) (157/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 90.0 in stage 7.0 (TID 494) in 727 ms on 10.60.24.13 (executor 1) (158/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 30.0 in stage 7.0 (TID 434) in 730 ms on 10.60.24.13 (executor 1) (159/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 46.0 in stage 7.0 (TID 450) in 730 ms on 10.60.24.13 (executor 1) (160/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 124.0 in stage 7.0 (TID 528) in 724 ms on 10.60.24.13 (executor 1) (161/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 118.0 in stage 7.0 (TID 522) in 724 ms on 10.60.24.13 (executor 1) (162/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 110.0 in stage 7.0 (TID 514) in 724 ms on 10.60.24.13 (executor 1) (163/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 108.0 in stage 7.0 (TID 512) in 729 ms on 10.60.24.13 (executor 1) (164/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 156.0 in stage 7.0 (TID 560) in 728 ms on 10.60.24.13 (executor 1) (165/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 50.0 in stage 7.0 (TID 454) in 736 ms on 10.60.24.13 (executor 1) (166/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 80.0 in stage 7.0 (TID 484) in 735 ms on 10.60.24.13 (executor 1) (167/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 38.0 in stage 7.0 (TID 442) in 736 ms on 10.60.24.13 (executor 1) (168/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 62.0 in stage 7.0 (TID 466) in 735 ms on 10.60.24.13 (executor 1) (169/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 120.0 in stage 7.0 (TID 524) in 730 ms on 10.60.24.13 (executor 1) (170/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 32.0 in stage 7.0 (TID 436) in 737 ms on 10.60.24.13 (executor 1) (171/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 152.0 in stage 7.0 (TID 556) in 729 ms on 10.60.24.13 (executor 1) (172/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 180.0 in stage 7.0 (TID 584) in 729 ms on 10.60.24.13 (executor 1) (173/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 44.0 in stage 7.0 (TID 448) in 738 ms on 10.60.24.13 (executor 1) (174/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 52.0 in stage 7.0 (TID 456) in 738 ms on 10.60.24.13 (executor 1) (175/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 74.0 in stage 7.0 (TID 478) in 738 ms on 10.60.24.13 (executor 1) (176/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 60.0 in stage 7.0 (TID 464) in 738 ms on 10.60.24.13 (executor 1) (177/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 172.0 in stage 7.0 (TID 576) in 730 ms on 10.60.24.13 (executor 1) (178/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 134.0 in stage 7.0 (TID 538) in 731 ms on 10.60.24.13 (executor 1) (179/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 130.0 in stage 7.0 (TID 534) in 731 ms on 10.60.24.13 (executor 1) (180/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 14.0 in stage 7.0 (TID 418) in 740 ms on 10.60.24.13 (executor 1) (181/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 42.0 in stage 7.0 (TID 446) in 739 ms on 10.60.24.13 (executor 1) (182/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 136.0 in stage 7.0 (TID 540) in 731 ms on 10.60.24.13 (executor 1) (183/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 84.0 in stage 7.0 (TID 488) in 737 ms on 10.60.24.13 (executor 1) (184/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 56.0 in stage 7.0 (TID 460) in 739 ms on 10.60.24.13 (executor 1) (185/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 150.0 in stage 7.0 (TID 554) in 732 ms on 10.60.24.13 (executor 1) (186/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 54.0 in stage 7.0 (TID 458) in 740 ms on 10.60.24.13 (executor 1) (187/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 96.0 in stage 7.0 (TID 500) in 737 ms on 10.60.24.13 (executor 1) (188/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 166.0 in stage 7.0 (TID 570) in 731 ms on 10.60.24.13 (executor 1) (189/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 34.0 in stage 7.0 (TID 438) in 740 ms on 10.60.24.13 (executor 1) (190/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 78.0 in stage 7.0 (TID 482) in 739 ms on 10.60.24.13 (executor 1) (191/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 36.0 in stage 7.0 (TID 440) in 740 ms on 10.60.24.13 (executor 1) (192/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 112.0 in stage 7.0 (TID 516) in 733 ms on 10.60.24.13 (executor 1) (193/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 160.0 in stage 7.0 (TID 564) in 731 ms on 10.60.24.13 (executor 1) (194/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 158.0 in stage 7.0 (TID 562) in 732 ms on 10.60.24.13 (executor 1) (195/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 146.0 in stage 7.0 (TID 550) in 733 ms on 10.60.24.13 (executor 1) (196/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 106.0 in stage 7.0 (TID 510) in 735 ms on 10.60.24.13 (executor 1) (197/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 24.0 in stage 7.0 (TID 428) in 742 ms on 10.60.24.13 (executor 1) (198/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 114.0 in stage 7.0 (TID 518) in 734 ms on 10.60.24.13 (executor 1) (199/200) 23/06/28 07:48:49 INFO TaskSetManager: Finished task 138.0 in stage 7.0 (TID 542) in 733 ms on 10.60.24.13 (executor 1) (200/200) 23/06/28 07:48:49 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:49 INFO DAGScheduler: ShuffleMapStage 7 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.779 s 23/06/28 07:48:49 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:49 INFO DAGScheduler: running: Set() 23/06/28 07:48:49 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:49 INFO DAGScheduler: failed: Set() 23/06/28 07:48:49 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:49 INFO SparkContext: Starting job: count at VacuumCommand.scala:188 23/06/28 07:48:49 INFO DAGScheduler: Got job 5 (count at VacuumCommand.scala:188) with 1 output partitions 23/06/28 07:48:49 INFO DAGScheduler: Final stage: ResultStage 11 (count at VacuumCommand.scala:188) 23/06/28 07:48:49 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 10) 23/06/28 07:48:49 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:49 INFO DAGScheduler: Submitting ResultStage 11 (MapPartitionsRDD[46] at count at VacuumCommand.scala:188), which has no missing parents 23/06/28 07:48:49 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:49 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:49 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:49 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:49 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 11 (MapPartitionsRDD[46] at count at VacuumCommand.scala:188) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:49 INFO TaskSchedulerImpl: Adding task set 11.0 with 1 tasks resource profile 0 23/06/28 07:48:49 INFO FairSchedulableBuilder: Added task set TaskSet_11.0 tasks to pool 7670323585580043917 23/06/28 07:48:49 INFO TaskSetManager: Starting task 0.0 in stage 11.0 (TID 604) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_16 stored as values in memory (estimated size 129.8 KiB, free 750.7 GiB) 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 21.3 KiB, free 750.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on 10.60.24.15:40507 (size: 21.3 KiB, free: 750.7 GiB) 23/06/28 07:48:49 INFO SparkContext: Created broadcast 16 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on 10.60.24.10:41041 (size: 21.3 KiB, free: 487.7 GiB) 23/06/28 07:48:49 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 4 to 10.60.24.10:44288 23/06/28 07:48:49 INFO TaskSetManager: Finished task 0.0 in stage 11.0 (TID 604) in 67 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:49 INFO TaskSchedulerImpl: Removed TaskSet 11.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:49 INFO DAGScheduler: ResultStage 11 (count at VacuumCommand.scala:188) finished in 0.070 s 23/06/28 07:48:49 INFO DAGScheduler: Job 5 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:49 INFO TaskSchedulerImpl: Killing all running tasks in stage 11: Stage finished 23/06/28 07:48:49 INFO DAGScheduler: Job 5 finished: count at VacuumCommand.scala:188, took 0.076030 s 23/06/28 07:48:49 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:49 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:49 INFO CodeGenerator: Code generated in 5.782954 ms 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_17 stored as values in memory (estimated size 5.8 KiB, free 750.7 GiB) 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 2.6 KiB, free 750.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on 10.60.24.15:40507 (size: 2.6 KiB, free: 750.7 GiB) 23/06/28 07:48:49 INFO SparkContext: Created broadcast 17 from first at VacuumCommand.scala:223 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_18 stored as values in memory (estimated size 3.4 KiB, free 750.7 GiB) 23/06/28 07:48:49 INFO MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 1650.0 B, free 750.7 GiB) 23/06/28 07:48:49 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on 10.60.24.15:40507 (size: 1650.0 B, free: 750.7 GiB) 23/06/28 07:48:49 INFO SparkContext: Created broadcast 18 from first at VacuumCommand.scala:223 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_19 stored as values in memory (estimated size 39.0 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 12.2 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on 10.60.24.15:40507 (size: 12.2 KiB, free: 750.7 GiB) 23/06/28 07:48:50 INFO SparkContext: Created broadcast 19 from first at VacuumCommand.scala:223 23/06/28 07:48:50 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:50 INFO CodeGenerator: Code generated in 5.141581 ms 23/06/28 07:48:50 INFO CodeGenerator: Code generated in 34.706712 ms 23/06/28 07:48:50 INFO DAGScheduler: Registering RDD 52 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 5 23/06/28 07:48:50 INFO DAGScheduler: Got map stage job 6 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:50 INFO DAGScheduler: Final stage: ShuffleMapStage 13 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 12) 23/06/28 07:48:50 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:50 INFO DAGScheduler: Submitting ShuffleMapStage 13 (MapPartitionsRDD[52] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:50 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:50 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:50 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:50 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:50 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 13 (MapPartitionsRDD[52] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:50 INFO TaskSchedulerImpl: Adding task set 13.0 with 1 tasks resource profile 0 23/06/28 07:48:50 INFO FairSchedulableBuilder: Added task set TaskSet_13.0 tasks to pool 7670323585580043917 23/06/28 07:48:50 INFO TaskSetManager: Starting task 0.0 in stage 13.0 (TID 605) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_20 stored as values in memory (estimated size 471.4 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO CodeGenerator: Code generated in 244.454064 ms 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 119.3 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on 10.60.24.15:40507 (size: 119.3 KiB, free: 750.7 GiB) 23/06/28 07:48:50 INFO SparkContext: Created broadcast 20 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on 10.60.24.13:44035 (size: 119.3 KiB, free: 487.7 GiB) 23/06/28 07:48:50 INFO CodeGenerator: Code generated in 7.953097 ms 23/06/28 07:48:50 INFO DAGScheduler: Registering RDD 60 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 6 23/06/28 07:48:50 INFO DAGScheduler: Got map stage job 7 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 200 output partitions 23/06/28 07:48:50 INFO DAGScheduler: Final stage: ShuffleMapStage 16 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:50 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 15) 23/06/28 07:48:50 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:50 INFO DAGScheduler: Submitting ShuffleMapStage 16 (MapPartitionsRDD[60] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:50 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:50 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:50 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:50 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:50 INFO DAGScheduler: Submitting 200 missing tasks from ShuffleMapStage 16 (MapPartitionsRDD[60] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:50 INFO TaskSchedulerImpl: Adding task set 16.0 with 200 tasks resource profile 0 23/06/28 07:48:50 INFO FairSchedulableBuilder: Added task set TaskSet_16.0 tasks to pool 7670323585580043917 23/06/28 07:48:50 INFO TaskSetManager: Starting task 0.0 in stage 16.0 (TID 606) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 1.0 in stage 16.0 (TID 607) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 2.0 in stage 16.0 (TID 608) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 3.0 in stage 16.0 (TID 609) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 4.0 in stage 16.0 (TID 610) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 5.0 in stage 16.0 (TID 611) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 6.0 in stage 16.0 (TID 612) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 7.0 in stage 16.0 (TID 613) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 8.0 in stage 16.0 (TID 614) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 9.0 in stage 16.0 (TID 615) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 10.0 in stage 16.0 (TID 616) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 11.0 in stage 16.0 (TID 617) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 12.0 in stage 16.0 (TID 618) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 13.0 in stage 16.0 (TID 619) (10.60.24.10, executor 0, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 14.0 in stage 16.0 (TID 620) (10.60.24.13, executor 1, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 15.0 in stage 16.0 (TID 621) (10.60.24.10, executor 0, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 16.0 in stage 16.0 (TID 622) (10.60.24.13, executor 1, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 17.0 in stage 16.0 (TID 623) (10.60.24.10, executor 0, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 18.0 in stage 16.0 (TID 624) (10.60.24.13, executor 1, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 19.0 in stage 16.0 (TID 625) (10.60.24.10, executor 0, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 20.0 in stage 16.0 (TID 626) (10.60.24.13, executor 1, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 21.0 in stage 16.0 (TID 627) (10.60.24.10, executor 0, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 22.0 in stage 16.0 (TID 628) (10.60.24.13, executor 1, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 23.0 in stage 16.0 (TID 629) (10.60.24.10, executor 0, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 24.0 in stage 16.0 (TID 630) (10.60.24.13, executor 1, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 25.0 in stage 16.0 (TID 631) (10.60.24.10, executor 0, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 26.0 in stage 16.0 (TID 632) (10.60.24.13, executor 1, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 27.0 in stage 16.0 (TID 633) (10.60.24.10, executor 0, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 28.0 in stage 16.0 (TID 634) (10.60.24.13, executor 1, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 29.0 in stage 16.0 (TID 635) (10.60.24.10, executor 0, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 30.0 in stage 16.0 (TID 636) (10.60.24.13, executor 1, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 31.0 in stage 16.0 (TID 637) (10.60.24.10, executor 0, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 32.0 in stage 16.0 (TID 638) (10.60.24.13, executor 1, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 33.0 in stage 16.0 (TID 639) (10.60.24.10, executor 0, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 34.0 in stage 16.0 (TID 640) (10.60.24.13, executor 1, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 35.0 in stage 16.0 (TID 641) (10.60.24.10, executor 0, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 36.0 in stage 16.0 (TID 642) (10.60.24.13, executor 1, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 37.0 in stage 16.0 (TID 643) (10.60.24.10, executor 0, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 38.0 in stage 16.0 (TID 644) (10.60.24.13, executor 1, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 39.0 in stage 16.0 (TID 645) (10.60.24.10, executor 0, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 40.0 in stage 16.0 (TID 646) (10.60.24.13, executor 1, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 41.0 in stage 16.0 (TID 647) (10.60.24.10, executor 0, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 42.0 in stage 16.0 (TID 648) (10.60.24.13, executor 1, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 43.0 in stage 16.0 (TID 649) (10.60.24.10, executor 0, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 44.0 in stage 16.0 (TID 650) (10.60.24.13, executor 1, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 45.0 in stage 16.0 (TID 651) (10.60.24.10, executor 0, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 46.0 in stage 16.0 (TID 652) (10.60.24.13, executor 1, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 47.0 in stage 16.0 (TID 653) (10.60.24.10, executor 0, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 48.0 in stage 16.0 (TID 654) (10.60.24.13, executor 1, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 49.0 in stage 16.0 (TID 655) (10.60.24.10, executor 0, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 50.0 in stage 16.0 (TID 656) (10.60.24.13, executor 1, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 51.0 in stage 16.0 (TID 657) (10.60.24.10, executor 0, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 52.0 in stage 16.0 (TID 658) (10.60.24.13, executor 1, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 53.0 in stage 16.0 (TID 659) (10.60.24.10, executor 0, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 54.0 in stage 16.0 (TID 660) (10.60.24.13, executor 1, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 55.0 in stage 16.0 (TID 661) (10.60.24.10, executor 0, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 56.0 in stage 16.0 (TID 662) (10.60.24.13, executor 1, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 57.0 in stage 16.0 (TID 663) (10.60.24.10, executor 0, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 58.0 in stage 16.0 (TID 664) (10.60.24.13, executor 1, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 59.0 in stage 16.0 (TID 665) (10.60.24.10, executor 0, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 60.0 in stage 16.0 (TID 666) (10.60.24.13, executor 1, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 61.0 in stage 16.0 (TID 667) (10.60.24.10, executor 0, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 62.0 in stage 16.0 (TID 668) (10.60.24.13, executor 1, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 63.0 in stage 16.0 (TID 669) (10.60.24.10, executor 0, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 64.0 in stage 16.0 (TID 670) (10.60.24.13, executor 1, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 65.0 in stage 16.0 (TID 671) (10.60.24.10, executor 0, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 66.0 in stage 16.0 (TID 672) (10.60.24.13, executor 1, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 67.0 in stage 16.0 (TID 673) (10.60.24.10, executor 0, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 68.0 in stage 16.0 (TID 674) (10.60.24.13, executor 1, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 69.0 in stage 16.0 (TID 675) (10.60.24.10, executor 0, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 70.0 in stage 16.0 (TID 676) (10.60.24.13, executor 1, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 71.0 in stage 16.0 (TID 677) (10.60.24.10, executor 0, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 72.0 in stage 16.0 (TID 678) (10.60.24.13, executor 1, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 73.0 in stage 16.0 (TID 679) (10.60.24.10, executor 0, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 74.0 in stage 16.0 (TID 680) (10.60.24.13, executor 1, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 75.0 in stage 16.0 (TID 681) (10.60.24.10, executor 0, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 76.0 in stage 16.0 (TID 682) (10.60.24.13, executor 1, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 77.0 in stage 16.0 (TID 683) (10.60.24.10, executor 0, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 78.0 in stage 16.0 (TID 684) (10.60.24.13, executor 1, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 79.0 in stage 16.0 (TID 685) (10.60.24.10, executor 0, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 80.0 in stage 16.0 (TID 686) (10.60.24.13, executor 1, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 81.0 in stage 16.0 (TID 687) (10.60.24.10, executor 0, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 82.0 in stage 16.0 (TID 688) (10.60.24.13, executor 1, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 83.0 in stage 16.0 (TID 689) (10.60.24.10, executor 0, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 84.0 in stage 16.0 (TID 690) (10.60.24.13, executor 1, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 85.0 in stage 16.0 (TID 691) (10.60.24.10, executor 0, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 86.0 in stage 16.0 (TID 692) (10.60.24.13, executor 1, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 87.0 in stage 16.0 (TID 693) (10.60.24.10, executor 0, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 88.0 in stage 16.0 (TID 694) (10.60.24.13, executor 1, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 89.0 in stage 16.0 (TID 695) (10.60.24.10, executor 0, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 90.0 in stage 16.0 (TID 696) (10.60.24.13, executor 1, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 91.0 in stage 16.0 (TID 697) (10.60.24.10, executor 0, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 92.0 in stage 16.0 (TID 698) (10.60.24.13, executor 1, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 93.0 in stage 16.0 (TID 699) (10.60.24.10, executor 0, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 94.0 in stage 16.0 (TID 700) (10.60.24.13, executor 1, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 95.0 in stage 16.0 (TID 701) (10.60.24.10, executor 0, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 96.0 in stage 16.0 (TID 702) (10.60.24.13, executor 1, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 97.0 in stage 16.0 (TID 703) (10.60.24.10, executor 0, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 98.0 in stage 16.0 (TID 704) (10.60.24.13, executor 1, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 99.0 in stage 16.0 (TID 705) (10.60.24.10, executor 0, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 100.0 in stage 16.0 (TID 706) (10.60.24.13, executor 1, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 101.0 in stage 16.0 (TID 707) (10.60.24.10, executor 0, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 102.0 in stage 16.0 (TID 708) (10.60.24.13, executor 1, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 103.0 in stage 16.0 (TID 709) (10.60.24.10, executor 0, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 104.0 in stage 16.0 (TID 710) (10.60.24.13, executor 1, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 105.0 in stage 16.0 (TID 711) (10.60.24.10, executor 0, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 106.0 in stage 16.0 (TID 712) (10.60.24.13, executor 1, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 107.0 in stage 16.0 (TID 713) (10.60.24.10, executor 0, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 108.0 in stage 16.0 (TID 714) (10.60.24.13, executor 1, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 109.0 in stage 16.0 (TID 715) (10.60.24.10, executor 0, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 110.0 in stage 16.0 (TID 716) (10.60.24.13, executor 1, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 111.0 in stage 16.0 (TID 717) (10.60.24.10, executor 0, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 112.0 in stage 16.0 (TID 718) (10.60.24.13, executor 1, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 113.0 in stage 16.0 (TID 719) (10.60.24.10, executor 0, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 114.0 in stage 16.0 (TID 720) (10.60.24.13, executor 1, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 115.0 in stage 16.0 (TID 721) (10.60.24.10, executor 0, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 116.0 in stage 16.0 (TID 722) (10.60.24.13, executor 1, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 117.0 in stage 16.0 (TID 723) (10.60.24.10, executor 0, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 118.0 in stage 16.0 (TID 724) (10.60.24.13, executor 1, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 119.0 in stage 16.0 (TID 725) (10.60.24.10, executor 0, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 120.0 in stage 16.0 (TID 726) (10.60.24.13, executor 1, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 121.0 in stage 16.0 (TID 727) (10.60.24.10, executor 0, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 122.0 in stage 16.0 (TID 728) (10.60.24.13, executor 1, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 123.0 in stage 16.0 (TID 729) (10.60.24.10, executor 0, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 124.0 in stage 16.0 (TID 730) (10.60.24.13, executor 1, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 125.0 in stage 16.0 (TID 731) (10.60.24.10, executor 0, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 126.0 in stage 16.0 (TID 732) (10.60.24.13, executor 1, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 127.0 in stage 16.0 (TID 733) (10.60.24.10, executor 0, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 128.0 in stage 16.0 (TID 734) (10.60.24.13, executor 1, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 129.0 in stage 16.0 (TID 735) (10.60.24.10, executor 0, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 130.0 in stage 16.0 (TID 736) (10.60.24.13, executor 1, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 131.0 in stage 16.0 (TID 737) (10.60.24.10, executor 0, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 132.0 in stage 16.0 (TID 738) (10.60.24.13, executor 1, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 133.0 in stage 16.0 (TID 739) (10.60.24.10, executor 0, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 134.0 in stage 16.0 (TID 740) (10.60.24.13, executor 1, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 135.0 in stage 16.0 (TID 741) (10.60.24.10, executor 0, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 136.0 in stage 16.0 (TID 742) (10.60.24.13, executor 1, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 137.0 in stage 16.0 (TID 743) (10.60.24.10, executor 0, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 138.0 in stage 16.0 (TID 744) (10.60.24.13, executor 1, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 139.0 in stage 16.0 (TID 745) (10.60.24.10, executor 0, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 140.0 in stage 16.0 (TID 746) (10.60.24.13, executor 1, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 141.0 in stage 16.0 (TID 747) (10.60.24.10, executor 0, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 142.0 in stage 16.0 (TID 748) (10.60.24.13, executor 1, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 143.0 in stage 16.0 (TID 749) (10.60.24.10, executor 0, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 144.0 in stage 16.0 (TID 750) (10.60.24.13, executor 1, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 145.0 in stage 16.0 (TID 751) (10.60.24.10, executor 0, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 146.0 in stage 16.0 (TID 752) (10.60.24.13, executor 1, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 147.0 in stage 16.0 (TID 753) (10.60.24.10, executor 0, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 148.0 in stage 16.0 (TID 754) (10.60.24.13, executor 1, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 149.0 in stage 16.0 (TID 755) (10.60.24.10, executor 0, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 150.0 in stage 16.0 (TID 756) (10.60.24.13, executor 1, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 151.0 in stage 16.0 (TID 757) (10.60.24.10, executor 0, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 152.0 in stage 16.0 (TID 758) (10.60.24.13, executor 1, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 153.0 in stage 16.0 (TID 759) (10.60.24.10, executor 0, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 154.0 in stage 16.0 (TID 760) (10.60.24.13, executor 1, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 155.0 in stage 16.0 (TID 761) (10.60.24.10, executor 0, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 156.0 in stage 16.0 (TID 762) (10.60.24.13, executor 1, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 157.0 in stage 16.0 (TID 763) (10.60.24.10, executor 0, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 158.0 in stage 16.0 (TID 764) (10.60.24.13, executor 1, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 159.0 in stage 16.0 (TID 765) (10.60.24.10, executor 0, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 160.0 in stage 16.0 (TID 766) (10.60.24.13, executor 1, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 161.0 in stage 16.0 (TID 767) (10.60.24.10, executor 0, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 162.0 in stage 16.0 (TID 768) (10.60.24.13, executor 1, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 163.0 in stage 16.0 (TID 769) (10.60.24.10, executor 0, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 164.0 in stage 16.0 (TID 770) (10.60.24.13, executor 1, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 165.0 in stage 16.0 (TID 771) (10.60.24.10, executor 0, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 166.0 in stage 16.0 (TID 772) (10.60.24.13, executor 1, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 167.0 in stage 16.0 (TID 773) (10.60.24.10, executor 0, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 168.0 in stage 16.0 (TID 774) (10.60.24.13, executor 1, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 169.0 in stage 16.0 (TID 775) (10.60.24.10, executor 0, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 170.0 in stage 16.0 (TID 776) (10.60.24.13, executor 1, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 171.0 in stage 16.0 (TID 777) (10.60.24.10, executor 0, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 172.0 in stage 16.0 (TID 778) (10.60.24.13, executor 1, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 173.0 in stage 16.0 (TID 779) (10.60.24.10, executor 0, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 174.0 in stage 16.0 (TID 780) (10.60.24.13, executor 1, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 175.0 in stage 16.0 (TID 781) (10.60.24.10, executor 0, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 176.0 in stage 16.0 (TID 782) (10.60.24.13, executor 1, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 177.0 in stage 16.0 (TID 783) (10.60.24.10, executor 0, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 178.0 in stage 16.0 (TID 784) (10.60.24.13, executor 1, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 179.0 in stage 16.0 (TID 785) (10.60.24.10, executor 0, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 180.0 in stage 16.0 (TID 786) (10.60.24.13, executor 1, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 181.0 in stage 16.0 (TID 787) (10.60.24.10, executor 0, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 182.0 in stage 16.0 (TID 788) (10.60.24.13, executor 1, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 183.0 in stage 16.0 (TID 789) (10.60.24.10, executor 0, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 184.0 in stage 16.0 (TID 790) (10.60.24.13, executor 1, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 185.0 in stage 16.0 (TID 791) (10.60.24.10, executor 0, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 186.0 in stage 16.0 (TID 792) (10.60.24.13, executor 1, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 187.0 in stage 16.0 (TID 793) (10.60.24.10, executor 0, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 188.0 in stage 16.0 (TID 794) (10.60.24.13, executor 1, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 189.0 in stage 16.0 (TID 795) (10.60.24.10, executor 0, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO TaskSetManager: Starting task 191.0 in stage 16.0 (TID 796) (10.60.24.10, executor 0, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_21 stored as values in memory (estimated size 193.2 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 48.8 KiB, free 750.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on 10.60.24.15:40507 (size: 48.8 KiB, free: 750.7 GiB) 23/06/28 07:48:50 INFO SparkContext: Created broadcast 21 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on 10.60.24.10:41041 (size: 48.8 KiB, free: 487.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on 10.60.24.13:44035 (size: 48.8 KiB, free: 487.7 GiB) 23/06/28 07:48:50 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 10.60.24.13:58548 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on 10.60.24.13:44035 (size: 2.6 KiB, free: 487.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on 10.60.24.10:41041 (size: 2.6 KiB, free: 487.7 GiB) 23/06/28 07:48:50 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on 10.60.24.13:44035 (size: 12.2 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 190.0 in stage 16.0 (TID 797) (10.60.24.13, executor 1, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 56.0 in stage 16.0 (TID 662) in 673 ms on 10.60.24.13 (executor 1) (1/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 40.0 in stage 16.0 (TID 646) in 689 ms on 10.60.24.13 (executor 1) (2/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 4.0 in stage 16.0 (TID 610) in 726 ms on 10.60.24.13 (executor 1) (3/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 88.0 in stage 16.0 (TID 694) in 753 ms on 10.60.24.13 (executor 1) (4/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 188.0 in stage 16.0 (TID 794) in 752 ms on 10.60.24.13 (executor 1) (5/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 176.0 in stage 16.0 (TID 782) in 769 ms on 10.60.24.13 (executor 1) (6/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 192.0 in stage 16.0 (TID 798) (10.60.24.10, executor 0, partition 192, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 181.0 in stage 16.0 (TID 787) in 770 ms on 10.60.24.10 (executor 0) (7/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 110.0 in stage 16.0 (TID 716) in 772 ms on 10.60.24.13 (executor 1) (8/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 86.0 in stage 16.0 (TID 692) in 773 ms on 10.60.24.13 (executor 1) (9/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 84.0 in stage 16.0 (TID 690) in 774 ms on 10.60.24.13 (executor 1) (10/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 164.0 in stage 16.0 (TID 770) in 779 ms on 10.60.24.13 (executor 1) (11/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 193.0 in stage 16.0 (TID 799) (10.60.24.10, executor 0, partition 193, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 125.0 in stage 16.0 (TID 731) in 785 ms on 10.60.24.10 (executor 0) (12/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 78.0 in stage 16.0 (TID 684) in 788 ms on 10.60.24.13 (executor 1) (13/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 62.0 in stage 16.0 (TID 668) in 790 ms on 10.60.24.13 (executor 1) (14/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 76.0 in stage 16.0 (TID 682) in 790 ms on 10.60.24.13 (executor 1) (15/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 126.0 in stage 16.0 (TID 732) in 798 ms on 10.60.24.13 (executor 1) (16/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 52.0 in stage 16.0 (TID 658) in 815 ms on 10.60.24.13 (executor 1) (17/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 130.0 in stage 16.0 (TID 736) in 815 ms on 10.60.24.13 (executor 1) (18/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 172.0 in stage 16.0 (TID 778) in 813 ms on 10.60.24.13 (executor 1) (19/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 80.0 in stage 16.0 (TID 686) in 817 ms on 10.60.24.13 (executor 1) (20/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 128.0 in stage 16.0 (TID 734) in 820 ms on 10.60.24.13 (executor 1) (21/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 32.0 in stage 16.0 (TID 638) in 823 ms on 10.60.24.13 (executor 1) (22/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 194.0 in stage 16.0 (TID 800) (10.60.24.10, executor 0, partition 194, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 157.0 in stage 16.0 (TID 763) in 830 ms on 10.60.24.10 (executor 0) (23/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 46.0 in stage 16.0 (TID 652) in 844 ms on 10.60.24.13 (executor 1) (24/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 166.0 in stage 16.0 (TID 772) in 840 ms on 10.60.24.13 (executor 1) (25/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 195.0 in stage 16.0 (TID 801) (10.60.24.10, executor 0, partition 195, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 39.0 in stage 16.0 (TID 645) in 851 ms on 10.60.24.10 (executor 0) (26/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 196.0 in stage 16.0 (TID 802) (10.60.24.10, executor 0, partition 196, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 133.0 in stage 16.0 (TID 739) in 853 ms on 10.60.24.10 (executor 0) (27/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 197.0 in stage 16.0 (TID 803) (10.60.24.10, executor 0, partition 197, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 15.0 in stage 16.0 (TID 621) in 860 ms on 10.60.24.10 (executor 0) (28/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 198.0 in stage 16.0 (TID 804) (10.60.24.10, executor 0, partition 198, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 87.0 in stage 16.0 (TID 693) in 878 ms on 10.60.24.10 (executor 0) (29/200) 23/06/28 07:48:51 INFO TaskSetManager: Starting task 199.0 in stage 16.0 (TID 805) (10.60.24.10, executor 0, partition 199, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 117.0 in stage 16.0 (TID 723) in 881 ms on 10.60.24.10 (executor 0) (30/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 155.0 in stage 16.0 (TID 761) in 879 ms on 10.60.24.10 (executor 0) (31/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 13.0 in stage 16.0 (TID 619) in 890 ms on 10.60.24.10 (executor 0) (32/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 91.0 in stage 16.0 (TID 697) in 888 ms on 10.60.24.10 (executor 0) (33/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 75.0 in stage 16.0 (TID 681) in 889 ms on 10.60.24.10 (executor 0) (34/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 169.0 in stage 16.0 (TID 775) in 885 ms on 10.60.24.10 (executor 0) (35/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 159.0 in stage 16.0 (TID 765) in 885 ms on 10.60.24.10 (executor 0) (36/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 123.0 in stage 16.0 (TID 729) in 887 ms on 10.60.24.10 (executor 0) (37/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 121.0 in stage 16.0 (TID 727) in 887 ms on 10.60.24.10 (executor 0) (38/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 149.0 in stage 16.0 (TID 755) in 887 ms on 10.60.24.10 (executor 0) (39/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 21.0 in stage 16.0 (TID 627) in 891 ms on 10.60.24.10 (executor 0) (40/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 83.0 in stage 16.0 (TID 689) in 890 ms on 10.60.24.10 (executor 0) (41/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 9.0 in stage 16.0 (TID 615) in 892 ms on 10.60.24.10 (executor 0) (42/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 3.0 in stage 16.0 (TID 609) in 894 ms on 10.60.24.10 (executor 0) (43/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 105.0 in stage 16.0 (TID 711) in 891 ms on 10.60.24.10 (executor 0) (44/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 51.0 in stage 16.0 (TID 657) in 893 ms on 10.60.24.10 (executor 0) (45/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 143.0 in stage 16.0 (TID 749) in 889 ms on 10.60.24.10 (executor 0) (46/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 77.0 in stage 16.0 (TID 683) in 892 ms on 10.60.24.10 (executor 0) (47/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 93.0 in stage 16.0 (TID 699) in 891 ms on 10.60.24.10 (executor 0) (48/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 165.0 in stage 16.0 (TID 771) in 888 ms on 10.60.24.10 (executor 0) (49/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 29.0 in stage 16.0 (TID 635) in 894 ms on 10.60.24.10 (executor 0) (50/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 113.0 in stage 16.0 (TID 719) in 893 ms on 10.60.24.10 (executor 0) (51/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 55.0 in stage 16.0 (TID 661) in 895 ms on 10.60.24.10 (executor 0) (52/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 191.0 in stage 16.0 (TID 796) in 893 ms on 10.60.24.10 (executor 0) (53/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 17.0 in stage 16.0 (TID 623) in 901 ms on 10.60.24.10 (executor 0) (54/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 151.0 in stage 16.0 (TID 757) in 895 ms on 10.60.24.10 (executor 0) (55/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 107.0 in stage 16.0 (TID 713) in 899 ms on 10.60.24.10 (executor 0) (56/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 41.0 in stage 16.0 (TID 647) in 901 ms on 10.60.24.10 (executor 0) (57/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 163.0 in stage 16.0 (TID 769) in 896 ms on 10.60.24.10 (executor 0) (58/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 131.0 in stage 16.0 (TID 737) in 898 ms on 10.60.24.10 (executor 0) (59/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 31.0 in stage 16.0 (TID 637) in 901 ms on 10.60.24.10 (executor 0) (60/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 81.0 in stage 16.0 (TID 687) in 904 ms on 10.60.24.10 (executor 0) (61/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 19.0 in stage 16.0 (TID 625) in 908 ms on 10.60.24.10 (executor 0) (62/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 171.0 in stage 16.0 (TID 777) in 903 ms on 10.60.24.10 (executor 0) (63/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 109.0 in stage 16.0 (TID 715) in 906 ms on 10.60.24.10 (executor 0) (64/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 177.0 in stage 16.0 (TID 783) in 903 ms on 10.60.24.10 (executor 0) (65/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 187.0 in stage 16.0 (TID 793) in 902 ms on 10.60.24.10 (executor 0) (66/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 189.0 in stage 16.0 (TID 795) in 903 ms on 10.60.24.10 (executor 0) (67/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 37.0 in stage 16.0 (TID 643) in 910 ms on 10.60.24.10 (executor 0) (68/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 141.0 in stage 16.0 (TID 747) in 907 ms on 10.60.24.10 (executor 0) (69/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 119.0 in stage 16.0 (TID 725) in 907 ms on 10.60.24.10 (executor 0) (70/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 179.0 in stage 16.0 (TID 785) in 905 ms on 10.60.24.10 (executor 0) (71/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 183.0 in stage 16.0 (TID 789) in 907 ms on 10.60.24.10 (executor 0) (72/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 7.0 in stage 16.0 (TID 613) in 913 ms on 10.60.24.10 (executor 0) (73/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 153.0 in stage 16.0 (TID 759) in 907 ms on 10.60.24.10 (executor 0) (74/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 145.0 in stage 16.0 (TID 751) in 908 ms on 10.60.24.10 (executor 0) (75/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 23.0 in stage 16.0 (TID 629) in 913 ms on 10.60.24.10 (executor 0) (76/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 65.0 in stage 16.0 (TID 671) in 912 ms on 10.60.24.10 (executor 0) (77/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 67.0 in stage 16.0 (TID 673) in 912 ms on 10.60.24.10 (executor 0) (78/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 127.0 in stage 16.0 (TID 733) in 911 ms on 10.60.24.10 (executor 0) (79/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 161.0 in stage 16.0 (TID 767) in 910 ms on 10.60.24.10 (executor 0) (80/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 185.0 in stage 16.0 (TID 791) in 909 ms on 10.60.24.10 (executor 0) (81/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 175.0 in stage 16.0 (TID 781) in 910 ms on 10.60.24.10 (executor 0) (82/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 57.0 in stage 16.0 (TID 663) in 914 ms on 10.60.24.10 (executor 0) (83/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 43.0 in stage 16.0 (TID 649) in 915 ms on 10.60.24.10 (executor 0) (84/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 27.0 in stage 16.0 (TID 633) in 916 ms on 10.60.24.10 (executor 0) (85/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 79.0 in stage 16.0 (TID 685) in 915 ms on 10.60.24.10 (executor 0) (86/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 147.0 in stage 16.0 (TID 753) in 912 ms on 10.60.24.10 (executor 0) (87/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 99.0 in stage 16.0 (TID 705) in 915 ms on 10.60.24.10 (executor 0) (88/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 25.0 in stage 16.0 (TID 631) in 917 ms on 10.60.24.10 (executor 0) (89/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 85.0 in stage 16.0 (TID 691) in 915 ms on 10.60.24.10 (executor 0) (90/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 5.0 in stage 16.0 (TID 611) in 919 ms on 10.60.24.10 (executor 0) (91/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 53.0 in stage 16.0 (TID 659) in 916 ms on 10.60.24.10 (executor 0) (92/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 129.0 in stage 16.0 (TID 735) in 914 ms on 10.60.24.10 (executor 0) (93/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 1.0 in stage 16.0 (TID 607) in 920 ms on 10.60.24.10 (executor 0) (94/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 135.0 in stage 16.0 (TID 741) in 915 ms on 10.60.24.10 (executor 0) (95/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 47.0 in stage 16.0 (TID 653) in 919 ms on 10.60.24.10 (executor 0) (96/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 167.0 in stage 16.0 (TID 773) in 914 ms on 10.60.24.10 (executor 0) (97/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 61.0 in stage 16.0 (TID 667) in 918 ms on 10.60.24.10 (executor 0) (98/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 115.0 in stage 16.0 (TID 721) in 916 ms on 10.60.24.10 (executor 0) (99/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 97.0 in stage 16.0 (TID 703) in 918 ms on 10.60.24.10 (executor 0) (100/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 49.0 in stage 16.0 (TID 655) in 920 ms on 10.60.24.10 (executor 0) (101/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 35.0 in stage 16.0 (TID 641) in 920 ms on 10.60.24.10 (executor 0) (102/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 59.0 in stage 16.0 (TID 665) in 919 ms on 10.60.24.10 (executor 0) (103/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 137.0 in stage 16.0 (TID 743) in 917 ms on 10.60.24.10 (executor 0) (104/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 103.0 in stage 16.0 (TID 709) in 919 ms on 10.60.24.10 (executor 0) (105/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 71.0 in stage 16.0 (TID 677) in 920 ms on 10.60.24.10 (executor 0) (106/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 11.0 in stage 16.0 (TID 617) in 922 ms on 10.60.24.10 (executor 0) (107/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 111.0 in stage 16.0 (TID 717) in 918 ms on 10.60.24.10 (executor 0) (108/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 173.0 in stage 16.0 (TID 779) in 917 ms on 10.60.24.10 (executor 0) (109/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 63.0 in stage 16.0 (TID 669) in 921 ms on 10.60.24.10 (executor 0) (110/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 101.0 in stage 16.0 (TID 707) in 920 ms on 10.60.24.10 (executor 0) (111/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 139.0 in stage 16.0 (TID 745) in 920 ms on 10.60.24.10 (executor 0) (112/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 69.0 in stage 16.0 (TID 675) in 922 ms on 10.60.24.10 (executor 0) (113/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 95.0 in stage 16.0 (TID 701) in 921 ms on 10.60.24.10 (executor 0) (114/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 89.0 in stage 16.0 (TID 695) in 921 ms on 10.60.24.10 (executor 0) (115/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 45.0 in stage 16.0 (TID 651) in 923 ms on 10.60.24.10 (executor 0) (116/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 33.0 in stage 16.0 (TID 639) in 924 ms on 10.60.24.10 (executor 0) (117/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 73.0 in stage 16.0 (TID 679) in 923 ms on 10.60.24.10 (executor 0) (118/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 114.0 in stage 16.0 (TID 720) in 940 ms on 10.60.24.13 (executor 1) (119/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 156.0 in stage 16.0 (TID 762) in 938 ms on 10.60.24.13 (executor 1) (120/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 152.0 in stage 16.0 (TID 758) in 938 ms on 10.60.24.13 (executor 1) (121/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 96.0 in stage 16.0 (TID 702) in 942 ms on 10.60.24.13 (executor 1) (122/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 36.0 in stage 16.0 (TID 642) in 947 ms on 10.60.24.13 (executor 1) (123/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 16.0 in stage 16.0 (TID 622) in 948 ms on 10.60.24.13 (executor 1) (124/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 22.0 in stage 16.0 (TID 628) in 947 ms on 10.60.24.13 (executor 1) (125/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 44.0 in stage 16.0 (TID 650) in 949 ms on 10.60.24.13 (executor 1) (126/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 106.0 in stage 16.0 (TID 712) in 947 ms on 10.60.24.13 (executor 1) (127/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 28.0 in stage 16.0 (TID 634) in 949 ms on 10.60.24.13 (executor 1) (128/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 64.0 in stage 16.0 (TID 670) in 948 ms on 10.60.24.13 (executor 1) (129/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 112.0 in stage 16.0 (TID 718) in 947 ms on 10.60.24.13 (executor 1) (130/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 162.0 in stage 16.0 (TID 768) in 945 ms on 10.60.24.13 (executor 1) (131/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 2.0 in stage 16.0 (TID 608) in 952 ms on 10.60.24.13 (executor 1) (132/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 10.0 in stage 16.0 (TID 616) in 951 ms on 10.60.24.13 (executor 1) (133/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 48.0 in stage 16.0 (TID 654) in 950 ms on 10.60.24.13 (executor 1) (134/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 72.0 in stage 16.0 (TID 678) in 951 ms on 10.60.24.13 (executor 1) (135/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 104.0 in stage 16.0 (TID 710) in 950 ms on 10.60.24.13 (executor 1) (136/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 174.0 in stage 16.0 (TID 780) in 947 ms on 10.60.24.13 (executor 1) (137/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 0.0 in stage 16.0 (TID 606) in 954 ms on 10.60.24.13 (executor 1) (138/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 120.0 in stage 16.0 (TID 726) in 949 ms on 10.60.24.13 (executor 1) (139/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 122.0 in stage 16.0 (TID 728) in 949 ms on 10.60.24.13 (executor 1) (140/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 170.0 in stage 16.0 (TID 776) in 947 ms on 10.60.24.13 (executor 1) (141/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 50.0 in stage 16.0 (TID 656) in 953 ms on 10.60.24.13 (executor 1) (142/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 168.0 in stage 16.0 (TID 774) in 948 ms on 10.60.24.13 (executor 1) (143/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 34.0 in stage 16.0 (TID 640) in 953 ms on 10.60.24.13 (executor 1) (144/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 18.0 in stage 16.0 (TID 624) in 954 ms on 10.60.24.13 (executor 1) (145/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 150.0 in stage 16.0 (TID 756) in 949 ms on 10.60.24.13 (executor 1) (146/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 102.0 in stage 16.0 (TID 708) in 951 ms on 10.60.24.13 (executor 1) (147/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 6.0 in stage 16.0 (TID 612) in 955 ms on 10.60.24.13 (executor 1) (148/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 8.0 in stage 16.0 (TID 614) in 954 ms on 10.60.24.13 (executor 1) (149/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 30.0 in stage 16.0 (TID 636) in 953 ms on 10.60.24.13 (executor 1) (150/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 14.0 in stage 16.0 (TID 620) in 954 ms on 10.60.24.13 (executor 1) (151/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 58.0 in stage 16.0 (TID 664) in 953 ms on 10.60.24.13 (executor 1) (152/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 154.0 in stage 16.0 (TID 760) in 949 ms on 10.60.24.13 (executor 1) (153/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 24.0 in stage 16.0 (TID 630) in 954 ms on 10.60.24.13 (executor 1) (154/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 144.0 in stage 16.0 (TID 750) in 950 ms on 10.60.24.13 (executor 1) (155/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 124.0 in stage 16.0 (TID 730) in 951 ms on 10.60.24.13 (executor 1) (156/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 74.0 in stage 16.0 (TID 680) in 953 ms on 10.60.24.13 (executor 1) (157/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 100.0 in stage 16.0 (TID 706) in 952 ms on 10.60.24.13 (executor 1) (158/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 60.0 in stage 16.0 (TID 666) in 953 ms on 10.60.24.13 (executor 1) (159/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 178.0 in stage 16.0 (TID 784) in 952 ms on 10.60.24.13 (executor 1) (160/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 146.0 in stage 16.0 (TID 752) in 953 ms on 10.60.24.13 (executor 1) (161/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 42.0 in stage 16.0 (TID 648) in 957 ms on 10.60.24.13 (executor 1) (162/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 82.0 in stage 16.0 (TID 688) in 957 ms on 10.60.24.13 (executor 1) (163/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 142.0 in stage 16.0 (TID 748) in 954 ms on 10.60.24.13 (executor 1) (164/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 20.0 in stage 16.0 (TID 626) in 959 ms on 10.60.24.13 (executor 1) (165/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 132.0 in stage 16.0 (TID 738) in 955 ms on 10.60.24.13 (executor 1) (166/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 118.0 in stage 16.0 (TID 724) in 955 ms on 10.60.24.13 (executor 1) (167/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 116.0 in stage 16.0 (TID 722) in 955 ms on 10.60.24.13 (executor 1) (168/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 158.0 in stage 16.0 (TID 764) in 953 ms on 10.60.24.13 (executor 1) (169/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 182.0 in stage 16.0 (TID 788) in 954 ms on 10.60.24.13 (executor 1) (170/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 134.0 in stage 16.0 (TID 740) in 956 ms on 10.60.24.13 (executor 1) (171/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 98.0 in stage 16.0 (TID 704) in 957 ms on 10.60.24.13 (executor 1) (172/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 180.0 in stage 16.0 (TID 786) in 954 ms on 10.60.24.13 (executor 1) (173/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 94.0 in stage 16.0 (TID 700) in 957 ms on 10.60.24.13 (executor 1) (174/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 38.0 in stage 16.0 (TID 644) in 959 ms on 10.60.24.13 (executor 1) (175/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 92.0 in stage 16.0 (TID 698) in 958 ms on 10.60.24.13 (executor 1) (176/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 186.0 in stage 16.0 (TID 792) in 954 ms on 10.60.24.13 (executor 1) (177/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 140.0 in stage 16.0 (TID 746) in 957 ms on 10.60.24.13 (executor 1) (178/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 160.0 in stage 16.0 (TID 766) in 955 ms on 10.60.24.13 (executor 1) (179/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 136.0 in stage 16.0 (TID 742) in 957 ms on 10.60.24.13 (executor 1) (180/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 148.0 in stage 16.0 (TID 754) in 956 ms on 10.60.24.13 (executor 1) (181/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 90.0 in stage 16.0 (TID 696) in 958 ms on 10.60.24.13 (executor 1) (182/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 12.0 in stage 16.0 (TID 618) in 961 ms on 10.60.24.13 (executor 1) (183/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 138.0 in stage 16.0 (TID 744) in 957 ms on 10.60.24.13 (executor 1) (184/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 54.0 in stage 16.0 (TID 660) in 959 ms on 10.60.24.13 (executor 1) (185/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 66.0 in stage 16.0 (TID 672) in 959 ms on 10.60.24.13 (executor 1) (186/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 108.0 in stage 16.0 (TID 714) in 958 ms on 10.60.24.13 (executor 1) (187/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 184.0 in stage 16.0 (TID 790) in 955 ms on 10.60.24.13 (executor 1) (188/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 26.0 in stage 16.0 (TID 632) in 960 ms on 10.60.24.13 (executor 1) (189/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 70.0 in stage 16.0 (TID 676) in 959 ms on 10.60.24.13 (executor 1) (190/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 68.0 in stage 16.0 (TID 674) in 960 ms on 10.60.24.13 (executor 1) (191/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 193.0 in stage 16.0 (TID 799) in 179 ms on 10.60.24.10 (executor 0) (192/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 199.0 in stage 16.0 (TID 805) in 84 ms on 10.60.24.10 (executor 0) (193/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 192.0 in stage 16.0 (TID 798) in 194 ms on 10.60.24.10 (executor 0) (194/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 197.0 in stage 16.0 (TID 803) in 110 ms on 10.60.24.10 (executor 0) (195/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 198.0 in stage 16.0 (TID 804) in 90 ms on 10.60.24.10 (executor 0) (196/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 196.0 in stage 16.0 (TID 802) in 116 ms on 10.60.24.10 (executor 0) (197/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 194.0 in stage 16.0 (TID 800) in 142 ms on 10.60.24.10 (executor 0) (198/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 195.0 in stage 16.0 (TID 801) in 130 ms on 10.60.24.10 (executor 0) (199/200) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 190.0 in stage 16.0 (TID 797) in 316 ms on 10.60.24.13 (executor 1) (200/200) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Removed TaskSet 16.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:51 INFO DAGScheduler: ShuffleMapStage 16 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 1.021 s 23/06/28 07:48:51 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:51 INFO DAGScheduler: running: Set(ShuffleMapStage 13) 23/06/28 07:48:51 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:51 INFO DAGScheduler: failed: Set() 23/06/28 07:48:51 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on 10.60.24.13:44035 (size: 14.1 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added rdd_23_0 in memory on 10.60.24.13:44035 (size: 2.3 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on 10.60.24.13:44035 (size: 12.2 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on 10.60.24.13:44035 (size: 1650.0 B, free: 487.7 GiB) 23/06/28 07:48:51 INFO TaskSetManager: Finished task 0.0 in stage 13.0 (TID 605) in 1189 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Removed TaskSet 13.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:51 INFO DAGScheduler: ShuffleMapStage 13 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 1.401 s 23/06/28 07:48:51 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:51 INFO DAGScheduler: running: Set() 23/06/28 07:48:51 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:51 INFO DAGScheduler: failed: Set() 23/06/28 07:48:51 INFO ShufflePartitionsUtil: For shuffle(5), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:51 INFO DAGScheduler: Registering RDD 62 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 7 23/06/28 07:48:51 INFO DAGScheduler: Got map stage job 8 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:51 INFO DAGScheduler: Final stage: ShuffleMapStage 19 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:51 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 18) 23/06/28 07:48:51 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:51 INFO DAGScheduler: Submitting ShuffleMapStage 19 (MapPartitionsRDD[62] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:51 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:51 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:51 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:51 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:51 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 19 (MapPartitionsRDD[62] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Adding task set 19.0 with 1 tasks resource profile 0 23/06/28 07:48:51 INFO FairSchedulableBuilder: Added task set TaskSet_19.0 tasks to pool 7670323585580043917 23/06/28 07:48:51 INFO TaskSetManager: Starting task 0.0 in stage 19.0 (TID 806) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_22 stored as values in memory (estimated size 127.6 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_22_piece0 stored as bytes in memory (estimated size 20.5 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on 10.60.24.15:40507 (size: 20.5 KiB, free: 750.7 GiB) 23/06/28 07:48:51 INFO SparkContext: Created broadcast 22 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on 10.60.24.13:44035 (size: 20.5 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 5 to 10.60.24.13:58548 23/06/28 07:48:51 INFO TaskSetManager: Finished task 0.0 in stage 19.0 (TID 806) in 26 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Removed TaskSet 19.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:51 INFO DAGScheduler: ShuffleMapStage 19 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.030 s 23/06/28 07:48:51 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:51 INFO DAGScheduler: running: Set() 23/06/28 07:48:51 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:51 INFO DAGScheduler: failed: Set() 23/06/28 07:48:51 INFO ShufflePartitionsUtil: For shuffle(6), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:51 INFO HashAggregateExec: spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current version of codegened fast hashmap does not support this aggregate. 23/06/28 07:48:51 INFO CodeGenerator: Code generated in 15.300873 ms 23/06/28 07:48:51 INFO DAGScheduler: Registering RDD 70 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 8 23/06/28 07:48:51 INFO DAGScheduler: Got map stage job 9 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:51 INFO DAGScheduler: Final stage: ShuffleMapStage 26 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:51 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 24, ShuffleMapStage 25) 23/06/28 07:48:51 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:51 INFO DAGScheduler: Submitting ShuffleMapStage 26 (MapPartitionsRDD[70] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:51 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:51 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:51 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:51 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:51 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 26 (MapPartitionsRDD[70] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Adding task set 26.0 with 1 tasks resource profile 0 23/06/28 07:48:51 INFO FairSchedulableBuilder: Added task set TaskSet_26.0 tasks to pool 7670323585580043917 23/06/28 07:48:51 INFO TaskSetManager: Starting task 0.0 in stage 26.0 (TID 807) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_23 stored as values in memory (estimated size 477.5 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_23_piece0 stored as bytes in memory (estimated size 129.6 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on 10.60.24.15:40507 (size: 129.6 KiB, free: 750.7 GiB) 23/06/28 07:48:51 INFO SparkContext: Created broadcast 23 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on 10.60.24.13:44035 (size: 129.6 KiB, free: 487.7 GiB) 23/06/28 07:48:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 6 to 10.60.24.13:58548 23/06/28 07:48:51 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 7 to 10.60.24.13:58548 23/06/28 07:48:51 INFO TaskSetManager: Finished task 0.0 in stage 26.0 (TID 807) in 166 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Removed TaskSet 26.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:51 INFO DAGScheduler: ShuffleMapStage 26 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.202 s 23/06/28 07:48:51 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:51 INFO DAGScheduler: running: Set() 23/06/28 07:48:51 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:51 INFO DAGScheduler: failed: Set() 23/06/28 07:48:51 INFO CodeGenerator: Code generated in 7.737371 ms 23/06/28 07:48:51 INFO SparkContext: Starting job: first at VacuumCommand.scala:223 23/06/28 07:48:51 INFO DAGScheduler: Got job 10 (first at VacuumCommand.scala:223) with 1 output partitions 23/06/28 07:48:51 INFO DAGScheduler: Final stage: ResultStage 34 (first at VacuumCommand.scala:223) 23/06/28 07:48:51 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 33) 23/06/28 07:48:51 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:51 INFO DAGScheduler: Submitting ResultStage 34 (MapPartitionsRDD[72] at first at VacuumCommand.scala:223), which has no missing parents 23/06/28 07:48:51 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:51 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:51 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:51 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:51 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 34 (MapPartitionsRDD[72] at first at VacuumCommand.scala:223) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:51 INFO TaskSchedulerImpl: Adding task set 34.0 with 1 tasks resource profile 0 23/06/28 07:48:51 INFO FairSchedulableBuilder: Added task set TaskSet_34.0 tasks to pool 7670323585580043917 23/06/28 07:48:51 INFO TaskSetManager: Starting task 0.0 in stage 34.0 (TID 808) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_24 stored as values in memory (estimated size 130.1 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO MemoryStore: Block broadcast_24_piece0 stored as bytes in memory (estimated size 21.5 KiB, free 750.7 GiB) 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on 10.60.24.15:40507 (size: 21.5 KiB, free: 750.7 GiB) 23/06/28 07:48:51 INFO SparkContext: Created broadcast 24 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:51 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on 10.60.24.13:44035 (size: 21.5 KiB, free: 487.7 GiB) 23/06/28 07:48:52 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 8 to 10.60.24.13:58548 23/06/28 07:48:52 INFO TaskSetManager: Finished task 0.0 in stage 34.0 (TID 808) in 49 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:52 INFO TaskSchedulerImpl: Removed TaskSet 34.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:52 INFO DAGScheduler: ResultStage 34 (first at VacuumCommand.scala:223) finished in 0.051 s 23/06/28 07:48:52 INFO DAGScheduler: Job 10 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:52 INFO TaskSchedulerImpl: Killing all running tasks in stage 34: Stage finished 23/06/28 07:48:52 INFO DAGScheduler: Job 10 finished: first at VacuumCommand.scala:223, took 0.054004 s 23/06/28 07:48:52 INFO CodeGenerator: Code generated in 6.542703 ms 23/06/28 07:48:52 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:52 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_7_370064497) 23/06/28 07:48:52 INFO VacuumCommand: Deleting untracked files and empty directories in dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta. The amount of data to be deleted is 0 (in bytes) 23/06/28 07:48:52 WARN BlockManager: Asked to remove block spark_join_relation_7_370064497, which does not exist 23/06/28 07:48:52 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:52 INFO CodeGenerator: Code generated in 38.755423 ms 23/06/28 07:48:52 INFO DAGScheduler: Registering RDD 75 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 9 23/06/28 07:48:52 INFO DAGScheduler: Got map stage job 11 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:52 INFO DAGScheduler: Final stage: ShuffleMapStage 36 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:52 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 35) 23/06/28 07:48:52 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:52 INFO DAGScheduler: Submitting ShuffleMapStage 36 (MapPartitionsRDD[75] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:52 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:52 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:52 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:52 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:52 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 36 (MapPartitionsRDD[75] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:52 INFO TaskSchedulerImpl: Adding task set 36.0 with 1 tasks resource profile 0 23/06/28 07:48:52 INFO FairSchedulableBuilder: Added task set TaskSet_36.0 tasks to pool 7670323585580043917 23/06/28 07:48:52 INFO TaskSetManager: Starting task 0.0 in stage 36.0 (TID 809) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:52 INFO MemoryStore: Block broadcast_25 stored as values in memory (estimated size 473.1 KiB, free 750.7 GiB) 23/06/28 07:48:52 INFO MemoryStore: Block broadcast_25_piece0 stored as bytes in memory (estimated size 120.5 KiB, free 750.7 GiB) 23/06/28 07:48:52 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on 10.60.24.15:40507 (size: 120.5 KiB, free: 750.7 GiB) 23/06/28 07:48:52 INFO SparkContext: Created broadcast 25 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:52 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on 10.60.24.13:44035 (size: 120.5 KiB, free: 487.7 GiB) 23/06/28 07:48:52 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:53 INFO TaskSetManager: Finished task 0.0 in stage 36.0 (TID 809) in 631 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:53 INFO TaskSchedulerImpl: Removed TaskSet 36.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:53 INFO DAGScheduler: ShuffleMapStage 36 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.668 s 23/06/28 07:48:53 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:53 INFO DAGScheduler: running: Set() 23/06/28 07:48:53 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:53 INFO DAGScheduler: failed: Set() 23/06/28 07:48:53 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:53 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:48:53 INFO DAGScheduler: Got job 12 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:48:53 INFO DAGScheduler: Final stage: ResultStage 39 (first at Snapshot.scala:271) 23/06/28 07:48:53 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 38) 23/06/28 07:48:53 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:53 INFO DAGScheduler: Submitting ResultStage 39 (MapPartitionsRDD[77] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:48:53 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:53 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:53 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:53 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:53 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 39 (MapPartitionsRDD[77] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:53 INFO TaskSchedulerImpl: Adding task set 39.0 with 1 tasks resource profile 0 23/06/28 07:48:53 INFO FairSchedulableBuilder: Added task set TaskSet_39.0 tasks to pool 7670323585580043917 23/06/28 07:48:53 INFO TaskSetManager: Starting task 0.0 in stage 39.0 (TID 810) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:53 INFO MemoryStore: Block broadcast_26 stored as values in memory (estimated size 419.2 KiB, free 750.7 GiB) 23/06/28 07:48:53 INFO MemoryStore: Block broadcast_26_piece0 stored as bytes in memory (estimated size 110.0 KiB, free 750.7 GiB) 23/06/28 07:48:53 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on 10.60.24.15:40507 (size: 110.0 KiB, free: 750.7 GiB) 23/06/28 07:48:53 INFO SparkContext: Created broadcast 26 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:53 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on 10.60.24.10:41041 (size: 110.0 KiB, free: 487.7 GiB) 23/06/28 07:48:53 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 9 to 10.60.24.10:44288 23/06/28 07:48:54 INFO TaskSetManager: Finished task 0.0 in stage 39.0 (TID 810) in 1188 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:54 INFO TaskSchedulerImpl: Removed TaskSet 39.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:54 INFO DAGScheduler: ResultStage 39 (first at Snapshot.scala:271) finished in 1.239 s 23/06/28 07:48:54 INFO DAGScheduler: Job 12 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:54 INFO TaskSchedulerImpl: Killing all running tasks in stage 39: Stage finished 23/06/28 07:48:54 INFO DAGScheduler: Job 12 finished: first at Snapshot.scala:271, took 1.241550 s 23/06/28 07:48:54 INFO CodeGenerator: Code generated in 27.931027 ms 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_27 stored as values in memory (estimated size 3.2 KiB, free 750.7 GiB) 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_27_piece0 stored as bytes in memory (estimated size 1999.0 B, free 750.7 GiB) 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on 10.60.24.15:40507 (size: 1999.0 B, free: 750.7 GiB) 23/06/28 07:48:54 INFO SparkContext: Created broadcast 27 from count at VacuumCommandImplEdge.scala:195 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:54 INFO DAGScheduler: Registering RDD 83 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 10 23/06/28 07:48:54 INFO DAGScheduler: Got map stage job 13 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:54 INFO DAGScheduler: Final stage: ShuffleMapStage 41 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:54 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 40) 23/06/28 07:48:54 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:54 INFO DAGScheduler: Submitting ShuffleMapStage 41 (MapPartitionsRDD[83] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:54 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:54 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:54 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:54 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 41 (MapPartitionsRDD[83] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:54 INFO TaskSchedulerImpl: Adding task set 41.0 with 1 tasks resource profile 0 23/06/28 07:48:54 INFO FairSchedulableBuilder: Added task set TaskSet_41.0 tasks to pool 7670323585580043917 23/06/28 07:48:54 INFO TaskSetManager: Starting task 0.0 in stage 41.0 (TID 811) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_28 stored as values in memory (estimated size 471.4 KiB, free 750.7 GiB) 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 119.3 KiB, free 750.7 GiB) 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on 10.60.24.15:40507 (size: 119.3 KiB, free: 750.7 GiB) 23/06/28 07:48:54 INFO SparkContext: Created broadcast 28 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on 10.60.24.13:44035 (size: 119.3 KiB, free: 487.7 GiB) 23/06/28 07:48:54 INFO CodeGenerator: Code generated in 83.445693 ms 23/06/28 07:48:54 INFO DAGScheduler: Registering RDD 91 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 11 23/06/28 07:48:54 INFO DAGScheduler: Got map stage job 14 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 200 output partitions 23/06/28 07:48:54 INFO DAGScheduler: Final stage: ShuffleMapStage 44 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:54 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 43) 23/06/28 07:48:54 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:54 INFO DAGScheduler: Submitting ShuffleMapStage 44 (MapPartitionsRDD[91] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:54 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:54 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:54 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:54 INFO DAGScheduler: Submitting 200 missing tasks from ShuffleMapStage 44 (MapPartitionsRDD[91] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:54 INFO TaskSchedulerImpl: Adding task set 44.0 with 200 tasks resource profile 0 23/06/28 07:48:54 INFO FairSchedulableBuilder: Added task set TaskSet_44.0 tasks to pool 7670323585580043917 23/06/28 07:48:54 INFO TaskSetManager: Starting task 0.0 in stage 44.0 (TID 812) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 1.0 in stage 44.0 (TID 813) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 2.0 in stage 44.0 (TID 814) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 3.0 in stage 44.0 (TID 815) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 4.0 in stage 44.0 (TID 816) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 5.0 in stage 44.0 (TID 817) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 6.0 in stage 44.0 (TID 818) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 7.0 in stage 44.0 (TID 819) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 8.0 in stage 44.0 (TID 820) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 9.0 in stage 44.0 (TID 821) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 10.0 in stage 44.0 (TID 822) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 11.0 in stage 44.0 (TID 823) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 12.0 in stage 44.0 (TID 824) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 13.0 in stage 44.0 (TID 825) (10.60.24.10, executor 0, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 14.0 in stage 44.0 (TID 826) (10.60.24.13, executor 1, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 15.0 in stage 44.0 (TID 827) (10.60.24.10, executor 0, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 16.0 in stage 44.0 (TID 828) (10.60.24.13, executor 1, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 17.0 in stage 44.0 (TID 829) (10.60.24.10, executor 0, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 18.0 in stage 44.0 (TID 830) (10.60.24.13, executor 1, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 19.0 in stage 44.0 (TID 831) (10.60.24.10, executor 0, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 20.0 in stage 44.0 (TID 832) (10.60.24.13, executor 1, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 21.0 in stage 44.0 (TID 833) (10.60.24.10, executor 0, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 22.0 in stage 44.0 (TID 834) (10.60.24.13, executor 1, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 23.0 in stage 44.0 (TID 835) (10.60.24.10, executor 0, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 24.0 in stage 44.0 (TID 836) (10.60.24.13, executor 1, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 25.0 in stage 44.0 (TID 837) (10.60.24.10, executor 0, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 26.0 in stage 44.0 (TID 838) (10.60.24.13, executor 1, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 27.0 in stage 44.0 (TID 839) (10.60.24.10, executor 0, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 28.0 in stage 44.0 (TID 840) (10.60.24.13, executor 1, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 29.0 in stage 44.0 (TID 841) (10.60.24.10, executor 0, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 30.0 in stage 44.0 (TID 842) (10.60.24.13, executor 1, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 31.0 in stage 44.0 (TID 843) (10.60.24.10, executor 0, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 32.0 in stage 44.0 (TID 844) (10.60.24.13, executor 1, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 33.0 in stage 44.0 (TID 845) (10.60.24.10, executor 0, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 34.0 in stage 44.0 (TID 846) (10.60.24.13, executor 1, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 35.0 in stage 44.0 (TID 847) (10.60.24.10, executor 0, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 36.0 in stage 44.0 (TID 848) (10.60.24.13, executor 1, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 37.0 in stage 44.0 (TID 849) (10.60.24.10, executor 0, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 38.0 in stage 44.0 (TID 850) (10.60.24.13, executor 1, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 39.0 in stage 44.0 (TID 851) (10.60.24.10, executor 0, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 40.0 in stage 44.0 (TID 852) (10.60.24.13, executor 1, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 41.0 in stage 44.0 (TID 853) (10.60.24.10, executor 0, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 42.0 in stage 44.0 (TID 854) (10.60.24.13, executor 1, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 43.0 in stage 44.0 (TID 855) (10.60.24.10, executor 0, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 44.0 in stage 44.0 (TID 856) (10.60.24.13, executor 1, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 45.0 in stage 44.0 (TID 857) (10.60.24.10, executor 0, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 46.0 in stage 44.0 (TID 858) (10.60.24.13, executor 1, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 47.0 in stage 44.0 (TID 859) (10.60.24.10, executor 0, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 48.0 in stage 44.0 (TID 860) (10.60.24.13, executor 1, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 49.0 in stage 44.0 (TID 861) (10.60.24.10, executor 0, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 50.0 in stage 44.0 (TID 862) (10.60.24.13, executor 1, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 51.0 in stage 44.0 (TID 863) (10.60.24.10, executor 0, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 52.0 in stage 44.0 (TID 864) (10.60.24.13, executor 1, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 53.0 in stage 44.0 (TID 865) (10.60.24.10, executor 0, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 54.0 in stage 44.0 (TID 866) (10.60.24.13, executor 1, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 55.0 in stage 44.0 (TID 867) (10.60.24.10, executor 0, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 56.0 in stage 44.0 (TID 868) (10.60.24.13, executor 1, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 57.0 in stage 44.0 (TID 869) (10.60.24.10, executor 0, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 58.0 in stage 44.0 (TID 870) (10.60.24.13, executor 1, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 59.0 in stage 44.0 (TID 871) (10.60.24.10, executor 0, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 60.0 in stage 44.0 (TID 872) (10.60.24.13, executor 1, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 61.0 in stage 44.0 (TID 873) (10.60.24.10, executor 0, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 62.0 in stage 44.0 (TID 874) (10.60.24.13, executor 1, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 63.0 in stage 44.0 (TID 875) (10.60.24.10, executor 0, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 64.0 in stage 44.0 (TID 876) (10.60.24.13, executor 1, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 65.0 in stage 44.0 (TID 877) (10.60.24.10, executor 0, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 66.0 in stage 44.0 (TID 878) (10.60.24.13, executor 1, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 67.0 in stage 44.0 (TID 879) (10.60.24.10, executor 0, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 68.0 in stage 44.0 (TID 880) (10.60.24.13, executor 1, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 69.0 in stage 44.0 (TID 881) (10.60.24.10, executor 0, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 70.0 in stage 44.0 (TID 882) (10.60.24.13, executor 1, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 71.0 in stage 44.0 (TID 883) (10.60.24.10, executor 0, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 72.0 in stage 44.0 (TID 884) (10.60.24.13, executor 1, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 73.0 in stage 44.0 (TID 885) (10.60.24.10, executor 0, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 74.0 in stage 44.0 (TID 886) (10.60.24.13, executor 1, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 75.0 in stage 44.0 (TID 887) (10.60.24.10, executor 0, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 76.0 in stage 44.0 (TID 888) (10.60.24.13, executor 1, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 77.0 in stage 44.0 (TID 889) (10.60.24.10, executor 0, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 78.0 in stage 44.0 (TID 890) (10.60.24.13, executor 1, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 79.0 in stage 44.0 (TID 891) (10.60.24.10, executor 0, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 80.0 in stage 44.0 (TID 892) (10.60.24.13, executor 1, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 81.0 in stage 44.0 (TID 893) (10.60.24.10, executor 0, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 82.0 in stage 44.0 (TID 894) (10.60.24.13, executor 1, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 83.0 in stage 44.0 (TID 895) (10.60.24.10, executor 0, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 84.0 in stage 44.0 (TID 896) (10.60.24.13, executor 1, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 85.0 in stage 44.0 (TID 897) (10.60.24.10, executor 0, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 86.0 in stage 44.0 (TID 898) (10.60.24.13, executor 1, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 87.0 in stage 44.0 (TID 899) (10.60.24.10, executor 0, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 88.0 in stage 44.0 (TID 900) (10.60.24.13, executor 1, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 89.0 in stage 44.0 (TID 901) (10.60.24.10, executor 0, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 90.0 in stage 44.0 (TID 902) (10.60.24.13, executor 1, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 91.0 in stage 44.0 (TID 903) (10.60.24.10, executor 0, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 92.0 in stage 44.0 (TID 904) (10.60.24.13, executor 1, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 93.0 in stage 44.0 (TID 905) (10.60.24.10, executor 0, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 94.0 in stage 44.0 (TID 906) (10.60.24.13, executor 1, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 95.0 in stage 44.0 (TID 907) (10.60.24.10, executor 0, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 96.0 in stage 44.0 (TID 908) (10.60.24.13, executor 1, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 97.0 in stage 44.0 (TID 909) (10.60.24.10, executor 0, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 98.0 in stage 44.0 (TID 910) (10.60.24.13, executor 1, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 99.0 in stage 44.0 (TID 911) (10.60.24.10, executor 0, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 100.0 in stage 44.0 (TID 912) (10.60.24.13, executor 1, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 101.0 in stage 44.0 (TID 913) (10.60.24.10, executor 0, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 102.0 in stage 44.0 (TID 914) (10.60.24.13, executor 1, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 103.0 in stage 44.0 (TID 915) (10.60.24.10, executor 0, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 104.0 in stage 44.0 (TID 916) (10.60.24.13, executor 1, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 105.0 in stage 44.0 (TID 917) (10.60.24.10, executor 0, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 106.0 in stage 44.0 (TID 918) (10.60.24.13, executor 1, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 107.0 in stage 44.0 (TID 919) (10.60.24.10, executor 0, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 108.0 in stage 44.0 (TID 920) (10.60.24.13, executor 1, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 109.0 in stage 44.0 (TID 921) (10.60.24.10, executor 0, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 110.0 in stage 44.0 (TID 922) (10.60.24.13, executor 1, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 111.0 in stage 44.0 (TID 923) (10.60.24.10, executor 0, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 112.0 in stage 44.0 (TID 924) (10.60.24.13, executor 1, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 113.0 in stage 44.0 (TID 925) (10.60.24.10, executor 0, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 114.0 in stage 44.0 (TID 926) (10.60.24.13, executor 1, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 115.0 in stage 44.0 (TID 927) (10.60.24.10, executor 0, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 116.0 in stage 44.0 (TID 928) (10.60.24.13, executor 1, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 117.0 in stage 44.0 (TID 929) (10.60.24.10, executor 0, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 118.0 in stage 44.0 (TID 930) (10.60.24.13, executor 1, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 119.0 in stage 44.0 (TID 931) (10.60.24.10, executor 0, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 120.0 in stage 44.0 (TID 932) (10.60.24.13, executor 1, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 121.0 in stage 44.0 (TID 933) (10.60.24.10, executor 0, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 122.0 in stage 44.0 (TID 934) (10.60.24.13, executor 1, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 123.0 in stage 44.0 (TID 935) (10.60.24.10, executor 0, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 124.0 in stage 44.0 (TID 936) (10.60.24.13, executor 1, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 125.0 in stage 44.0 (TID 937) (10.60.24.10, executor 0, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 126.0 in stage 44.0 (TID 938) (10.60.24.13, executor 1, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 127.0 in stage 44.0 (TID 939) (10.60.24.10, executor 0, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 128.0 in stage 44.0 (TID 940) (10.60.24.13, executor 1, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 129.0 in stage 44.0 (TID 941) (10.60.24.10, executor 0, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 130.0 in stage 44.0 (TID 942) (10.60.24.13, executor 1, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 131.0 in stage 44.0 (TID 943) (10.60.24.10, executor 0, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 132.0 in stage 44.0 (TID 944) (10.60.24.13, executor 1, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 133.0 in stage 44.0 (TID 945) (10.60.24.10, executor 0, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 134.0 in stage 44.0 (TID 946) (10.60.24.13, executor 1, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 135.0 in stage 44.0 (TID 947) (10.60.24.10, executor 0, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 136.0 in stage 44.0 (TID 948) (10.60.24.13, executor 1, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 137.0 in stage 44.0 (TID 949) (10.60.24.10, executor 0, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 138.0 in stage 44.0 (TID 950) (10.60.24.13, executor 1, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 139.0 in stage 44.0 (TID 951) (10.60.24.10, executor 0, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 140.0 in stage 44.0 (TID 952) (10.60.24.13, executor 1, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 141.0 in stage 44.0 (TID 953) (10.60.24.10, executor 0, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 142.0 in stage 44.0 (TID 954) (10.60.24.13, executor 1, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 143.0 in stage 44.0 (TID 955) (10.60.24.10, executor 0, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 144.0 in stage 44.0 (TID 956) (10.60.24.13, executor 1, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 145.0 in stage 44.0 (TID 957) (10.60.24.10, executor 0, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 146.0 in stage 44.0 (TID 958) (10.60.24.13, executor 1, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 147.0 in stage 44.0 (TID 959) (10.60.24.10, executor 0, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 148.0 in stage 44.0 (TID 960) (10.60.24.13, executor 1, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 149.0 in stage 44.0 (TID 961) (10.60.24.10, executor 0, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 150.0 in stage 44.0 (TID 962) (10.60.24.13, executor 1, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 151.0 in stage 44.0 (TID 963) (10.60.24.10, executor 0, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 152.0 in stage 44.0 (TID 964) (10.60.24.13, executor 1, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 153.0 in stage 44.0 (TID 965) (10.60.24.10, executor 0, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 154.0 in stage 44.0 (TID 966) (10.60.24.13, executor 1, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 155.0 in stage 44.0 (TID 967) (10.60.24.10, executor 0, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 156.0 in stage 44.0 (TID 968) (10.60.24.13, executor 1, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 157.0 in stage 44.0 (TID 969) (10.60.24.10, executor 0, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 158.0 in stage 44.0 (TID 970) (10.60.24.13, executor 1, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 159.0 in stage 44.0 (TID 971) (10.60.24.10, executor 0, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 160.0 in stage 44.0 (TID 972) (10.60.24.13, executor 1, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 161.0 in stage 44.0 (TID 973) (10.60.24.10, executor 0, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 162.0 in stage 44.0 (TID 974) (10.60.24.13, executor 1, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 163.0 in stage 44.0 (TID 975) (10.60.24.10, executor 0, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 164.0 in stage 44.0 (TID 976) (10.60.24.13, executor 1, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 165.0 in stage 44.0 (TID 977) (10.60.24.10, executor 0, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 166.0 in stage 44.0 (TID 978) (10.60.24.13, executor 1, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 167.0 in stage 44.0 (TID 979) (10.60.24.10, executor 0, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 168.0 in stage 44.0 (TID 980) (10.60.24.13, executor 1, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 169.0 in stage 44.0 (TID 981) (10.60.24.10, executor 0, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 170.0 in stage 44.0 (TID 982) (10.60.24.13, executor 1, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 171.0 in stage 44.0 (TID 983) (10.60.24.10, executor 0, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 172.0 in stage 44.0 (TID 984) (10.60.24.13, executor 1, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 173.0 in stage 44.0 (TID 985) (10.60.24.10, executor 0, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 174.0 in stage 44.0 (TID 986) (10.60.24.13, executor 1, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 175.0 in stage 44.0 (TID 987) (10.60.24.10, executor 0, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 176.0 in stage 44.0 (TID 988) (10.60.24.13, executor 1, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 177.0 in stage 44.0 (TID 989) (10.60.24.10, executor 0, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 178.0 in stage 44.0 (TID 990) (10.60.24.13, executor 1, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 179.0 in stage 44.0 (TID 991) (10.60.24.10, executor 0, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 180.0 in stage 44.0 (TID 992) (10.60.24.13, executor 1, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 181.0 in stage 44.0 (TID 993) (10.60.24.10, executor 0, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 182.0 in stage 44.0 (TID 994) (10.60.24.13, executor 1, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 183.0 in stage 44.0 (TID 995) (10.60.24.10, executor 0, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 184.0 in stage 44.0 (TID 996) (10.60.24.13, executor 1, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 185.0 in stage 44.0 (TID 997) (10.60.24.10, executor 0, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 186.0 in stage 44.0 (TID 998) (10.60.24.13, executor 1, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 187.0 in stage 44.0 (TID 999) (10.60.24.10, executor 0, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 188.0 in stage 44.0 (TID 1000) (10.60.24.13, executor 1, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 189.0 in stage 44.0 (TID 1001) (10.60.24.10, executor 0, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 191.0 in stage 44.0 (TID 1002) (10.60.24.10, executor 0, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_29 stored as values in memory (estimated size 190.5 KiB, free 750.7 GiB) 23/06/28 07:48:54 INFO MemoryStore: Block broadcast_29_piece0 stored as bytes in memory (estimated size 47.9 KiB, free 750.7 GiB) 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on 10.60.24.15:40507 (size: 47.9 KiB, free: 750.7 GiB) 23/06/28 07:48:54 INFO SparkContext: Created broadcast 29 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on 10.60.24.10:41041 (size: 47.9 KiB, free: 487.7 GiB) 23/06/28 07:48:54 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on 10.60.24.13:44035 (size: 47.9 KiB, free: 487.7 GiB) 23/06/28 07:48:54 INFO TaskSetManager: Starting task 190.0 in stage 44.0 (TID 1003) (10.60.24.13, executor 1, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:54 INFO TaskSetManager: Finished task 0.0 in stage 41.0 (TID 811) in 55 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:54 INFO TaskSchedulerImpl: Removed TaskSet 41.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:54 INFO DAGScheduler: ShuffleMapStage 41 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.110 s 23/06/28 07:48:54 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:54 INFO DAGScheduler: running: Set(ShuffleMapStage 44) 23/06/28 07:48:54 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:54 INFO DAGScheduler: failed: Set() 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:54 INFO ShufflePartitionsUtil: For shuffle(10), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:54 INFO DAGScheduler: Registering RDD 93 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 12 23/06/28 07:48:54 INFO DAGScheduler: Got map stage job 15 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:54 INFO DAGScheduler: Final stage: ShuffleMapStage 47 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:54 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 46) 23/06/28 07:48:54 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:54 INFO DAGScheduler: Submitting ShuffleMapStage 47 (MapPartitionsRDD[93] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:54 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:54 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:54 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:54 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:54 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 47 (MapPartitionsRDD[93] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:54 INFO TaskSchedulerImpl: Adding task set 47.0 with 1 tasks resource profile 0 23/06/28 07:48:54 INFO FairSchedulableBuilder: Added task set TaskSet_47.0 tasks to pool 7670323585580043917 23/06/28 07:48:55 INFO TaskSetManager: Starting task 0.0 in stage 47.0 (TID 1004) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 154.0 in stage 44.0 (TID 966) in 364 ms on 10.60.24.13 (executor 1) (1/200) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_30 stored as values in memory (estimated size 127.6 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 100.0 in stage 44.0 (TID 912) in 366 ms on 10.60.24.13 (executor 1) (2/200) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_30_piece0 stored as bytes in memory (estimated size 20.5 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on 10.60.24.15:40507 (size: 20.5 KiB, free: 750.7 GiB) 23/06/28 07:48:55 INFO SparkContext: Created broadcast 30 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:55 INFO TaskSetManager: Finished task 152.0 in stage 44.0 (TID 964) in 370 ms on 10.60.24.13 (executor 1) (3/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 16.0 in stage 44.0 (TID 828) in 375 ms on 10.60.24.13 (executor 1) (4/200) 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on 10.60.24.13:44035 (size: 20.5 KiB, free: 487.7 GiB) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 14.0 in stage 44.0 (TID 826) in 411 ms on 10.60.24.13 (executor 1) (5/200) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 192.0 in stage 44.0 (TID 1005) (10.60.24.10, executor 0, partition 192, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 193.0 in stage 44.0 (TID 1006) (10.60.24.10, executor 0, partition 193, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 115.0 in stage 44.0 (TID 927) in 545 ms on 10.60.24.10 (executor 0) (6/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 27.0 in stage 44.0 (TID 839) in 547 ms on 10.60.24.10 (executor 0) (7/200) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 194.0 in stage 44.0 (TID 1007) (10.60.24.10, executor 0, partition 194, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 195.0 in stage 44.0 (TID 1008) (10.60.24.10, executor 0, partition 195, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 196.0 in stage 44.0 (TID 1009) (10.60.24.10, executor 0, partition 196, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 197.0 in stage 44.0 (TID 1010) (10.60.24.10, executor 0, partition 197, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 198.0 in stage 44.0 (TID 1011) (10.60.24.10, executor 0, partition 198, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 79.0 in stage 44.0 (TID 891) in 547 ms on 10.60.24.10 (executor 0) (8/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 43.0 in stage 44.0 (TID 855) in 548 ms on 10.60.24.10 (executor 0) (9/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 53.0 in stage 44.0 (TID 865) in 548 ms on 10.60.24.10 (executor 0) (10/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 75.0 in stage 44.0 (TID 887) in 547 ms on 10.60.24.10 (executor 0) (11/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 87.0 in stage 44.0 (TID 899) in 547 ms on 10.60.24.10 (executor 0) (12/200) 23/06/28 07:48:55 INFO TaskSetManager: Starting task 199.0 in stage 44.0 (TID 1012) (10.60.24.10, executor 0, partition 199, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 169.0 in stage 44.0 (TID 981) in 545 ms on 10.60.24.10 (executor 0) (13/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 91.0 in stage 44.0 (TID 903) in 547 ms on 10.60.24.10 (executor 0) (14/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 65.0 in stage 44.0 (TID 877) in 547 ms on 10.60.24.10 (executor 0) (15/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 55.0 in stage 44.0 (TID 867) in 548 ms on 10.60.24.10 (executor 0) (16/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 177.0 in stage 44.0 (TID 989) in 546 ms on 10.60.24.10 (executor 0) (17/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 127.0 in stage 44.0 (TID 939) in 547 ms on 10.60.24.10 (executor 0) (18/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 174.0 in stage 44.0 (TID 986) in 546 ms on 10.60.24.13 (executor 1) (19/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 62.0 in stage 44.0 (TID 874) in 568 ms on 10.60.24.13 (executor 1) (20/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 24.0 in stage 44.0 (TID 836) in 569 ms on 10.60.24.13 (executor 1) (21/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 34.0 in stage 44.0 (TID 846) in 569 ms on 10.60.24.13 (executor 1) (22/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 168.0 in stage 44.0 (TID 980) in 586 ms on 10.60.24.13 (executor 1) (23/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 64.0 in stage 44.0 (TID 876) in 588 ms on 10.60.24.13 (executor 1) (24/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 50.0 in stage 44.0 (TID 862) in 609 ms on 10.60.24.13 (executor 1) (25/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 92.0 in stage 44.0 (TID 904) in 609 ms on 10.60.24.13 (executor 1) (26/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 142.0 in stage 44.0 (TID 954) in 608 ms on 10.60.24.13 (executor 1) (27/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 164.0 in stage 44.0 (TID 976) in 607 ms on 10.60.24.13 (executor 1) (28/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 120.0 in stage 44.0 (TID 932) in 612 ms on 10.60.24.13 (executor 1) (29/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 146.0 in stage 44.0 (TID 958) in 612 ms on 10.60.24.13 (executor 1) (30/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 48.0 in stage 44.0 (TID 860) in 614 ms on 10.60.24.13 (executor 1) (31/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 60.0 in stage 44.0 (TID 872) in 615 ms on 10.60.24.13 (executor 1) (32/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 108.0 in stage 44.0 (TID 920) in 613 ms on 10.60.24.13 (executor 1) (33/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 76.0 in stage 44.0 (TID 888) in 614 ms on 10.60.24.13 (executor 1) (34/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 38.0 in stage 44.0 (TID 850) in 615 ms on 10.60.24.13 (executor 1) (35/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 2.0 in stage 44.0 (TID 814) in 616 ms on 10.60.24.13 (executor 1) (36/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 136.0 in stage 44.0 (TID 948) in 613 ms on 10.60.24.13 (executor 1) (37/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 20.0 in stage 44.0 (TID 832) in 616 ms on 10.60.24.13 (executor 1) (38/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 82.0 in stage 44.0 (TID 894) in 614 ms on 10.60.24.13 (executor 1) (39/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 138.0 in stage 44.0 (TID 950) in 613 ms on 10.60.24.13 (executor 1) (40/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 176.0 in stage 44.0 (TID 988) in 613 ms on 10.60.24.13 (executor 1) (41/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 30.0 in stage 44.0 (TID 842) in 616 ms on 10.60.24.13 (executor 1) (42/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 112.0 in stage 44.0 (TID 924) in 614 ms on 10.60.24.13 (executor 1) (43/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 124.0 in stage 44.0 (TID 936) in 614 ms on 10.60.24.13 (executor 1) (44/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 36.0 in stage 44.0 (TID 848) in 616 ms on 10.60.24.13 (executor 1) (45/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 74.0 in stage 44.0 (TID 886) in 615 ms on 10.60.24.13 (executor 1) (46/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 86.0 in stage 44.0 (TID 898) in 615 ms on 10.60.24.13 (executor 1) (47/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 126.0 in stage 44.0 (TID 938) in 614 ms on 10.60.24.13 (executor 1) (48/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 150.0 in stage 44.0 (TID 962) in 614 ms on 10.60.24.13 (executor 1) (49/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 18.0 in stage 44.0 (TID 830) in 617 ms on 10.60.24.13 (executor 1) (50/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 52.0 in stage 44.0 (TID 864) in 617 ms on 10.60.24.13 (executor 1) (51/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 28.0 in stage 44.0 (TID 840) in 617 ms on 10.60.24.13 (executor 1) (52/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 188.0 in stage 44.0 (TID 1000) in 614 ms on 10.60.24.13 (executor 1) (53/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 162.0 in stage 44.0 (TID 974) in 614 ms on 10.60.24.13 (executor 1) (54/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 114.0 in stage 44.0 (TID 926) in 615 ms on 10.60.24.13 (executor 1) (55/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 180.0 in stage 44.0 (TID 992) in 614 ms on 10.60.24.13 (executor 1) (56/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 102.0 in stage 44.0 (TID 914) in 616 ms on 10.60.24.13 (executor 1) (57/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 178.0 in stage 44.0 (TID 990) in 614 ms on 10.60.24.13 (executor 1) (58/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 116.0 in stage 44.0 (TID 928) in 615 ms on 10.60.24.13 (executor 1) (59/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 172.0 in stage 44.0 (TID 984) in 614 ms on 10.60.24.13 (executor 1) (60/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 98.0 in stage 44.0 (TID 910) in 617 ms on 10.60.24.13 (executor 1) (61/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 0.0 in stage 44.0 (TID 812) in 619 ms on 10.60.24.13 (executor 1) (62/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 148.0 in stage 44.0 (TID 960) in 616 ms on 10.60.24.13 (executor 1) (63/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 4.0 in stage 44.0 (TID 816) in 622 ms on 10.60.24.13 (executor 1) (64/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 12.0 in stage 44.0 (TID 824) in 622 ms on 10.60.24.13 (executor 1) (65/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 132.0 in stage 44.0 (TID 944) in 619 ms on 10.60.24.13 (executor 1) (66/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 118.0 in stage 44.0 (TID 930) in 619 ms on 10.60.24.13 (executor 1) (67/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 80.0 in stage 44.0 (TID 892) in 620 ms on 10.60.24.13 (executor 1) (68/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 122.0 in stage 44.0 (TID 934) in 619 ms on 10.60.24.13 (executor 1) (69/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 166.0 in stage 44.0 (TID 978) in 618 ms on 10.60.24.13 (executor 1) (70/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 130.0 in stage 44.0 (TID 942) in 619 ms on 10.60.24.13 (executor 1) (71/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 42.0 in stage 44.0 (TID 854) in 622 ms on 10.60.24.13 (executor 1) (72/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 54.0 in stage 44.0 (TID 866) in 622 ms on 10.60.24.13 (executor 1) (73/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 10.0 in stage 44.0 (TID 822) in 623 ms on 10.60.24.13 (executor 1) (74/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 104.0 in stage 44.0 (TID 916) in 621 ms on 10.60.24.13 (executor 1) (75/200) 23/06/28 07:48:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 10 to 10.60.24.13:58548 23/06/28 07:48:55 INFO TaskSetManager: Finished task 6.0 in stage 44.0 (TID 818) in 623 ms on 10.60.24.13 (executor 1) (76/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 66.0 in stage 44.0 (TID 878) in 624 ms on 10.60.24.13 (executor 1) (77/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 110.0 in stage 44.0 (TID 922) in 623 ms on 10.60.24.13 (executor 1) (78/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 160.0 in stage 44.0 (TID 972) in 622 ms on 10.60.24.13 (executor 1) (79/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 170.0 in stage 44.0 (TID 982) in 623 ms on 10.60.24.13 (executor 1) (80/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 32.0 in stage 44.0 (TID 844) in 626 ms on 10.60.24.13 (executor 1) (81/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 90.0 in stage 44.0 (TID 902) in 625 ms on 10.60.24.13 (executor 1) (82/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 26.0 in stage 44.0 (TID 838) in 626 ms on 10.60.24.13 (executor 1) (83/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 78.0 in stage 44.0 (TID 890) in 625 ms on 10.60.24.13 (executor 1) (84/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 72.0 in stage 44.0 (TID 884) in 626 ms on 10.60.24.13 (executor 1) (85/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 46.0 in stage 44.0 (TID 858) in 627 ms on 10.60.24.13 (executor 1) (86/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 134.0 in stage 44.0 (TID 946) in 625 ms on 10.60.24.13 (executor 1) (87/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 88.0 in stage 44.0 (TID 900) in 626 ms on 10.60.24.13 (executor 1) (88/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 40.0 in stage 44.0 (TID 852) in 627 ms on 10.60.24.13 (executor 1) (89/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 144.0 in stage 44.0 (TID 956) in 625 ms on 10.60.24.13 (executor 1) (90/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 96.0 in stage 44.0 (TID 908) in 627 ms on 10.60.24.13 (executor 1) (91/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 8.0 in stage 44.0 (TID 820) in 629 ms on 10.60.24.13 (executor 1) (92/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 58.0 in stage 44.0 (TID 870) in 628 ms on 10.60.24.13 (executor 1) (93/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 22.0 in stage 44.0 (TID 834) in 628 ms on 10.60.24.13 (executor 1) (94/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 61.0 in stage 44.0 (TID 873) in 629 ms on 10.60.24.10 (executor 0) (95/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 140.0 in stage 44.0 (TID 952) in 627 ms on 10.60.24.13 (executor 1) (96/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 29.0 in stage 44.0 (TID 841) in 629 ms on 10.60.24.10 (executor 0) (97/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 94.0 in stage 44.0 (TID 906) in 628 ms on 10.60.24.13 (executor 1) (98/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 106.0 in stage 44.0 (TID 918) in 628 ms on 10.60.24.13 (executor 1) (99/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 84.0 in stage 44.0 (TID 896) in 628 ms on 10.60.24.13 (executor 1) (100/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 121.0 in stage 44.0 (TID 933) in 627 ms on 10.60.24.10 (executor 0) (101/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 182.0 in stage 44.0 (TID 994) in 626 ms on 10.60.24.13 (executor 1) (102/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 186.0 in stage 44.0 (TID 998) in 627 ms on 10.60.24.13 (executor 1) (103/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 158.0 in stage 44.0 (TID 970) in 627 ms on 10.60.24.13 (executor 1) (104/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 156.0 in stage 44.0 (TID 968) in 631 ms on 10.60.24.13 (executor 1) (105/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 190.0 in stage 44.0 (TID 1003) in 615 ms on 10.60.24.13 (executor 1) (106/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 44.0 in stage 44.0 (TID 856) in 634 ms on 10.60.24.13 (executor 1) (107/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 56.0 in stage 44.0 (TID 868) in 634 ms on 10.60.24.13 (executor 1) (108/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 0.0 in stage 47.0 (TID 1004) in 271 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Removed TaskSet 47.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:55 INFO DAGScheduler: ShuffleMapStage 47 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.594 s 23/06/28 07:48:55 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:55 INFO DAGScheduler: running: Set(ShuffleMapStage 44) 23/06/28 07:48:55 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:55 INFO DAGScheduler: failed: Set() 23/06/28 07:48:55 INFO TaskSetManager: Finished task 184.0 in stage 44.0 (TID 996) in 636 ms on 10.60.24.13 (executor 1) (109/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 70.0 in stage 44.0 (TID 882) in 639 ms on 10.60.24.13 (executor 1) (110/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 68.0 in stage 44.0 (TID 880) in 639 ms on 10.60.24.13 (executor 1) (111/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 128.0 in stage 44.0 (TID 940) in 638 ms on 10.60.24.13 (executor 1) (112/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 137.0 in stage 44.0 (TID 949) in 677 ms on 10.60.24.10 (executor 0) (113/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 133.0 in stage 44.0 (TID 945) in 677 ms on 10.60.24.10 (executor 0) (114/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 103.0 in stage 44.0 (TID 915) in 678 ms on 10.60.24.10 (executor 0) (115/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 7.0 in stage 44.0 (TID 819) in 680 ms on 10.60.24.10 (executor 0) (116/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 3.0 in stage 44.0 (TID 815) in 680 ms on 10.60.24.10 (executor 0) (117/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 95.0 in stage 44.0 (TID 907) in 678 ms on 10.60.24.10 (executor 0) (118/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 179.0 in stage 44.0 (TID 991) in 676 ms on 10.60.24.10 (executor 0) (119/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 51.0 in stage 44.0 (TID 863) in 680 ms on 10.60.24.10 (executor 0) (120/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 19.0 in stage 44.0 (TID 831) in 681 ms on 10.60.24.10 (executor 0) (121/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 117.0 in stage 44.0 (TID 929) in 678 ms on 10.60.24.10 (executor 0) (122/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 125.0 in stage 44.0 (TID 937) in 678 ms on 10.60.24.10 (executor 0) (123/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 77.0 in stage 44.0 (TID 889) in 679 ms on 10.60.24.10 (executor 0) (124/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 111.0 in stage 44.0 (TID 923) in 678 ms on 10.60.24.10 (executor 0) (125/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 83.0 in stage 44.0 (TID 895) in 679 ms on 10.60.24.10 (executor 0) (126/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 155.0 in stage 44.0 (TID 967) in 679 ms on 10.60.24.10 (executor 0) (127/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 181.0 in stage 44.0 (TID 993) in 678 ms on 10.60.24.10 (executor 0) (128/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 15.0 in stage 44.0 (TID 827) in 682 ms on 10.60.24.10 (executor 0) (129/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 107.0 in stage 44.0 (TID 919) in 679 ms on 10.60.24.10 (executor 0) (130/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 183.0 in stage 44.0 (TID 995) in 678 ms on 10.60.24.10 (executor 0) (131/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 1.0 in stage 44.0 (TID 813) in 682 ms on 10.60.24.10 (executor 0) (132/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 67.0 in stage 44.0 (TID 879) in 680 ms on 10.60.24.10 (executor 0) (133/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 97.0 in stage 44.0 (TID 909) in 681 ms on 10.60.24.10 (executor 0) (134/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 85.0 in stage 44.0 (TID 897) in 681 ms on 10.60.24.10 (executor 0) (135/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 189.0 in stage 44.0 (TID 1001) in 679 ms on 10.60.24.10 (executor 0) (136/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 141.0 in stage 44.0 (TID 953) in 680 ms on 10.60.24.10 (executor 0) (137/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 109.0 in stage 44.0 (TID 921) in 680 ms on 10.60.24.10 (executor 0) (138/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 17.0 in stage 44.0 (TID 829) in 683 ms on 10.60.24.10 (executor 0) (139/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 131.0 in stage 44.0 (TID 943) in 680 ms on 10.60.24.10 (executor 0) (140/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 23.0 in stage 44.0 (TID 835) in 683 ms on 10.60.24.10 (executor 0) (141/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 99.0 in stage 44.0 (TID 911) in 682 ms on 10.60.24.10 (executor 0) (142/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 63.0 in stage 44.0 (TID 875) in 682 ms on 10.60.24.10 (executor 0) (143/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 31.0 in stage 44.0 (TID 843) in 683 ms on 10.60.24.10 (executor 0) (144/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 151.0 in stage 44.0 (TID 963) in 681 ms on 10.60.24.10 (executor 0) (145/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 41.0 in stage 44.0 (TID 853) in 684 ms on 10.60.24.10 (executor 0) (146/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 135.0 in stage 44.0 (TID 947) in 682 ms on 10.60.24.10 (executor 0) (147/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 113.0 in stage 44.0 (TID 925) in 682 ms on 10.60.24.10 (executor 0) (148/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 13.0 in stage 44.0 (TID 825) in 685 ms on 10.60.24.10 (executor 0) (149/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 5.0 in stage 44.0 (TID 817) in 686 ms on 10.60.24.10 (executor 0) (150/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 33.0 in stage 44.0 (TID 845) in 685 ms on 10.60.24.10 (executor 0) (151/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 149.0 in stage 44.0 (TID 961) in 683 ms on 10.60.24.10 (executor 0) (152/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 57.0 in stage 44.0 (TID 869) in 686 ms on 10.60.24.10 (executor 0) (153/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 39.0 in stage 44.0 (TID 851) in 686 ms on 10.60.24.10 (executor 0) (154/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 175.0 in stage 44.0 (TID 987) in 683 ms on 10.60.24.10 (executor 0) (155/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 37.0 in stage 44.0 (TID 849) in 686 ms on 10.60.24.10 (executor 0) (156/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 123.0 in stage 44.0 (TID 935) in 684 ms on 10.60.24.10 (executor 0) (157/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 25.0 in stage 44.0 (TID 837) in 687 ms on 10.60.24.10 (executor 0) (158/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 187.0 in stage 44.0 (TID 999) in 698 ms on 10.60.24.10 (executor 0) (159/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 49.0 in stage 44.0 (TID 861) in 702 ms on 10.60.24.10 (executor 0) (160/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 147.0 in stage 44.0 (TID 959) in 700 ms on 10.60.24.10 (executor 0) (161/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 165.0 in stage 44.0 (TID 977) in 699 ms on 10.60.24.10 (executor 0) (162/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 143.0 in stage 44.0 (TID 955) in 715 ms on 10.60.24.10 (executor 0) (163/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 139.0 in stage 44.0 (TID 951) in 715 ms on 10.60.24.10 (executor 0) (164/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 89.0 in stage 44.0 (TID 901) in 717 ms on 10.60.24.10 (executor 0) (165/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 71.0 in stage 44.0 (TID 883) in 717 ms on 10.60.24.10 (executor 0) (166/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 153.0 in stage 44.0 (TID 965) in 716 ms on 10.60.24.10 (executor 0) (167/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 69.0 in stage 44.0 (TID 881) in 717 ms on 10.60.24.10 (executor 0) (168/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 185.0 in stage 44.0 (TID 997) in 715 ms on 10.60.24.10 (executor 0) (169/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 45.0 in stage 44.0 (TID 857) in 718 ms on 10.60.24.10 (executor 0) (170/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 81.0 in stage 44.0 (TID 893) in 717 ms on 10.60.24.10 (executor 0) (171/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 59.0 in stage 44.0 (TID 871) in 718 ms on 10.60.24.10 (executor 0) (172/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 119.0 in stage 44.0 (TID 931) in 717 ms on 10.60.24.10 (executor 0) (173/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 9.0 in stage 44.0 (TID 821) in 720 ms on 10.60.24.10 (executor 0) (174/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 73.0 in stage 44.0 (TID 885) in 718 ms on 10.60.24.10 (executor 0) (175/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 129.0 in stage 44.0 (TID 941) in 717 ms on 10.60.24.10 (executor 0) (176/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 105.0 in stage 44.0 (TID 917) in 718 ms on 10.60.24.10 (executor 0) (177/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 35.0 in stage 44.0 (TID 847) in 719 ms on 10.60.24.10 (executor 0) (178/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 163.0 in stage 44.0 (TID 975) in 716 ms on 10.60.24.10 (executor 0) (179/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 173.0 in stage 44.0 (TID 985) in 716 ms on 10.60.24.10 (executor 0) (180/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 157.0 in stage 44.0 (TID 969) in 717 ms on 10.60.24.10 (executor 0) (181/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 145.0 in stage 44.0 (TID 957) in 718 ms on 10.60.24.10 (executor 0) (182/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 47.0 in stage 44.0 (TID 859) in 720 ms on 10.60.24.10 (executor 0) (183/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 93.0 in stage 44.0 (TID 905) in 719 ms on 10.60.24.10 (executor 0) (184/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 161.0 in stage 44.0 (TID 973) in 717 ms on 10.60.24.10 (executor 0) (185/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 11.0 in stage 44.0 (TID 823) in 721 ms on 10.60.24.10 (executor 0) (186/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 191.0 in stage 44.0 (TID 1002) in 722 ms on 10.60.24.10 (executor 0) (187/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 167.0 in stage 44.0 (TID 979) in 723 ms on 10.60.24.10 (executor 0) (188/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 101.0 in stage 44.0 (TID 913) in 725 ms on 10.60.24.10 (executor 0) (189/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 171.0 in stage 44.0 (TID 983) in 723 ms on 10.60.24.10 (executor 0) (190/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 21.0 in stage 44.0 (TID 833) in 727 ms on 10.60.24.10 (executor 0) (191/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 159.0 in stage 44.0 (TID 971) in 726 ms on 10.60.24.10 (executor 0) (192/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 194.0 in stage 44.0 (TID 1007) in 221 ms on 10.60.24.10 (executor 0) (193/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 192.0 in stage 44.0 (TID 1005) in 229 ms on 10.60.24.10 (executor 0) (194/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 197.0 in stage 44.0 (TID 1010) in 229 ms on 10.60.24.10 (executor 0) (195/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 199.0 in stage 44.0 (TID 1012) in 231 ms on 10.60.24.10 (executor 0) (196/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 196.0 in stage 44.0 (TID 1009) in 232 ms on 10.60.24.10 (executor 0) (197/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 193.0 in stage 44.0 (TID 1006) in 232 ms on 10.60.24.10 (executor 0) (198/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 195.0 in stage 44.0 (TID 1008) in 236 ms on 10.60.24.10 (executor 0) (199/200) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 198.0 in stage 44.0 (TID 1011) in 238 ms on 10.60.24.10 (executor 0) (200/200) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Removed TaskSet 44.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:55 INFO DAGScheduler: ShuffleMapStage 44 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.794 s 23/06/28 07:48:55 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:55 INFO DAGScheduler: running: Set() 23/06/28 07:48:55 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:55 INFO DAGScheduler: failed: Set() 23/06/28 07:48:55 INFO ShufflePartitionsUtil: For shuffle(11), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:55 INFO CodeGenerator: Code generated in 9.912975 ms 23/06/28 07:48:55 INFO HashAggregateExec: spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current version of codegened fast hashmap does not support this aggregate. 23/06/28 07:48:55 INFO CodeGenerator: Code generated in 10.430673 ms 23/06/28 07:48:55 INFO DAGScheduler: Registering RDD 103 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 13 23/06/28 07:48:55 INFO DAGScheduler: Got map stage job 16 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:55 INFO DAGScheduler: Final stage: ShuffleMapStage 54 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:55 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 52, ShuffleMapStage 53) 23/06/28 07:48:55 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:55 INFO DAGScheduler: Submitting ShuffleMapStage 54 (MapPartitionsRDD[103] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:55 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:55 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:55 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:55 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:55 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 54 (MapPartitionsRDD[103] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Adding task set 54.0 with 1 tasks resource profile 0 23/06/28 07:48:55 INFO FairSchedulableBuilder: Added task set TaskSet_54.0 tasks to pool 7670323585580043917 23/06/28 07:48:55 INFO TaskSetManager: Starting task 0.0 in stage 54.0 (TID 1013) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_31 stored as values in memory (estimated size 475.8 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_31_piece0 stored as bytes in memory (estimated size 128.6 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on 10.60.24.15:40507 (size: 128.6 KiB, free: 750.7 GiB) 23/06/28 07:48:55 INFO SparkContext: Created broadcast 31 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on 10.60.24.13:44035 (size: 128.6 KiB, free: 487.7 GiB) 23/06/28 07:48:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 11 to 10.60.24.13:58548 23/06/28 07:48:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 12 to 10.60.24.13:58548 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on 10.60.24.13:44035 (size: 1999.0 B, free: 487.7 GiB) 23/06/28 07:48:55 INFO TaskSetManager: Finished task 0.0 in stage 54.0 (TID 1013) in 127 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Removed TaskSet 54.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:55 INFO DAGScheduler: ShuffleMapStage 54 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.147 s 23/06/28 07:48:55 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:55 INFO DAGScheduler: running: Set() 23/06/28 07:48:55 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:55 INFO DAGScheduler: failed: Set() 23/06/28 07:48:55 INFO CodeGenerator: Code generated in 7.248317 ms 23/06/28 07:48:55 INFO SparkContext: Starting job: count at VacuumCommandImplEdge.scala:195 23/06/28 07:48:55 INFO DAGScheduler: Got job 17 (count at VacuumCommandImplEdge.scala:195) with 1 output partitions 23/06/28 07:48:55 INFO DAGScheduler: Final stage: ResultStage 62 (count at VacuumCommandImplEdge.scala:195) 23/06/28 07:48:55 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 61) 23/06/28 07:48:55 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:55 INFO DAGScheduler: Submitting ResultStage 62 (MapPartitionsRDD[105] at count at VacuumCommandImplEdge.scala:195), which has no missing parents 23/06/28 07:48:55 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:55 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:55 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:55 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:55 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 62 (MapPartitionsRDD[105] at count at VacuumCommandImplEdge.scala:195) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Adding task set 62.0 with 1 tasks resource profile 0 23/06/28 07:48:55 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 7670323585580043917, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 7670323585580043917. Created 7670323585580043917 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:48:55 INFO FairSchedulableBuilder: Added task set TaskSet_62.0 tasks to pool 7670323585580043917 23/06/28 07:48:55 INFO TaskSetManager: Starting task 0.0 in stage 62.0 (TID 1014) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_32 stored as values in memory (estimated size 129.8 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO MemoryStore: Block broadcast_32_piece0 stored as bytes in memory (estimated size 21.3 KiB, free 750.7 GiB) 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on 10.60.24.15:40507 (size: 21.3 KiB, free: 750.7 GiB) 23/06/28 07:48:55 INFO SparkContext: Created broadcast 32 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:55 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on 10.60.24.10:41041 (size: 21.3 KiB, free: 487.7 GiB) 23/06/28 07:48:55 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 13 to 10.60.24.10:44288 23/06/28 07:48:55 INFO TaskSetManager: Finished task 0.0 in stage 62.0 (TID 1014) in 33 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:55 INFO TaskSchedulerImpl: Removed TaskSet 62.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:55 INFO DAGScheduler: ResultStage 62 (count at VacuumCommandImplEdge.scala:195) finished in 0.036 s 23/06/28 07:48:55 INFO DAGScheduler: Job 17 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:55 INFO TaskSchedulerImpl: Killing all running tasks in stage 62: Stage finished 23/06/28 07:48:55 INFO DAGScheduler: Job 17 finished: count at VacuumCommandImplEdge.scala:195, took 0.038918 s 23/06/28 07:48:55 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:55 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_12_370064497) 23/06/28 07:48:55 WARN BlockManager: Asked to remove block spark_join_relation_12_370064497, which does not exist 23/06/28 07:48:55 INFO OptimisticTransaction: [tableId=32485164,txnId=3c8f5a73] Attempting to commit version 85 with 1 actions with SnapshotIsolation isolation level 23/06/28 07:48:55 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:48:55 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp 23/06/28 07:48:55 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp] Closing stream; size: 0 23/06/28 07:48:55 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:55 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp] Upload complete; size: 787 23/06/28 07:48:55 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json] Starting rename. Issuing rename operation. 23/06/28 07:48:55 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json] Rename successful. 23/06/28 07:48:55 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.json.831b075e-c2d6-4d52-9a48-f7e5396d32b0.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json 23/06/28 07:48:55 INFO OptimisticTransaction: [tableId=32485164,txnId=3c8f5a73] Incremental commit: starting with snapshot version 84 23/06/28 07:48:56 INFO DeltaLog: Loading version 85 starting from checkpoint version 80. 23/06/28 07:48:56 INFO SnapshotEdge: [tableId=32485164-a88a-4b48-ac48-c8898e21a3ee] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=85, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,85,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938535000), checksumOpt=Some(VersionChecksum(Some(3c8f5a73-768f-45bb-8a07-76d582cbf470),75338,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@6dfa8024,[J@79d5090c)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-d26d832b-f936-48d3-a0bb-bfcbed64446c.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75338,1687937150000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687937150000000, MIN_INSERTION_TIME -> 1687937150000000, MAX_INSERTION_TIME -> 1687937150000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_33 stored as values in memory (estimated size 508.7 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:48:56 INFO SparkContext: Created broadcast 33 from broadcast at DeltaLog.scala:634 23/06/28 07:48:56 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 27842) 23/06/28 07:48:56 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 5, totalFileSize: 6536) 23/06/28 07:48:56 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:48:56 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_34 stored as values in memory (estimated size 609.9 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_34_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:48:56 INFO SparkContext: Created broadcast 34 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:48:56 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:48:56 INFO DAGScheduler: Registering RDD 112 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 14 23/06/28 07:48:56 INFO DAGScheduler: Got map stage job 18 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 10 output partitions 23/06/28 07:48:56 INFO DAGScheduler: Final stage: ShuffleMapStage 63 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:56 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:48:56 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:56 INFO DAGScheduler: Submitting ShuffleMapStage 63 (MapPartitionsRDD[112] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:56 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:56 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:56 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:56 INFO DAGScheduler: Submitting 10 missing tasks from ShuffleMapStage 63 (MapPartitionsRDD[112] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 23/06/28 07:48:56 INFO TaskSchedulerImpl: Adding task set 63.0 with 10 tasks resource profile 0 23/06/28 07:48:56 INFO FairSchedulableBuilder: Added task set TaskSet_63.0 tasks to pool 7670323585580043917 23/06/28 07:48:56 INFO TaskSetManager: Starting task 0.0 in stage 63.0 (TID 1015) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 1.0 in stage 63.0 (TID 1016) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 2.0 in stage 63.0 (TID 1017) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 3.0 in stage 63.0 (TID 1018) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 4.0 in stage 63.0 (TID 1019) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 5.0 in stage 63.0 (TID 1020) (10.60.24.13, executor 1, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 6.0 in stage 63.0 (TID 1021) (10.60.24.10, executor 0, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 7.0 in stage 63.0 (TID 1022) (10.60.24.13, executor 1, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 8.0 in stage 63.0 (TID 1023) (10.60.24.10, executor 0, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO TaskSetManager: Starting task 9.0 in stage 63.0 (TID 1024) (10.60.24.13, executor 1, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_35 stored as values in memory (estimated size 324.3 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_35_piece0 stored as bytes in memory (estimated size 79.3 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on 10.60.24.15:40507 (size: 79.3 KiB, free: 750.7 GiB) 23/06/28 07:48:56 INFO SparkContext: Created broadcast 35 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on 10.60.24.10:41041 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on 10.60.24.13:44035 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 3.0 in stage 63.0 (TID 1018) in 22 ms on 10.60.24.13 (executor 1) (1/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 7.0 in stage 63.0 (TID 1022) in 22 ms on 10.60.24.13 (executor 1) (2/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 5.0 in stage 63.0 (TID 1020) in 22 ms on 10.60.24.13 (executor 1) (3/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 1.0 in stage 63.0 (TID 1016) in 22 ms on 10.60.24.13 (executor 1) (4/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 9.0 in stage 63.0 (TID 1024) in 22 ms on 10.60.24.13 (executor 1) (5/10) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 8.0 in stage 63.0 (TID 1023) in 24 ms on 10.60.24.10 (executor 0) (6/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 2.0 in stage 63.0 (TID 1017) in 24 ms on 10.60.24.10 (executor 0) (7/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 4.0 in stage 63.0 (TID 1019) in 25 ms on 10.60.24.10 (executor 0) (8/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 6.0 in stage 63.0 (TID 1021) in 25 ms on 10.60.24.10 (executor 0) (9/10) 23/06/28 07:48:56 INFO TaskSetManager: Finished task 0.0 in stage 63.0 (TID 1015) in 102 ms on 10.60.24.10 (executor 0) (10/10) 23/06/28 07:48:56 INFO TaskSchedulerImpl: Removed TaskSet 63.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:56 INFO DAGScheduler: ShuffleMapStage 63 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.115 s 23/06/28 07:48:56 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:56 INFO DAGScheduler: running: Set() 23/06/28 07:48:56 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:56 INFO DAGScheduler: failed: Set() 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:56 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":5103,"lifetime":235001},"10.60.24.13":{"diskUsage":0,"lifetime":235271}}, ioCacheNumScanTasks={"numLocalScanTasks":2,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:48:56 INFO CodeGenerator: Code generated in 25.057381 ms 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_36 stored as values in memory (estimated size 8.9 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_36_piece0 stored as bytes in memory (estimated size 4.5 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on 10.60.24.15:40507 (size: 4.5 KiB, free: 750.7 GiB) 23/06/28 07:48:56 INFO SparkContext: Created broadcast 36 from collect at Checksum.scala:561 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:56 INFO CodeGenerator: Code generated in 37.590233 ms 23/06/28 07:48:56 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:48:56 INFO DAGScheduler: Got job 19 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:48:56 INFO DAGScheduler: Final stage: ResultStage 65 (collect at Checksum.scala:561) 23/06/28 07:48:56 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 64) 23/06/28 07:48:56 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:56 INFO DAGScheduler: Submitting ResultStage 65 (MapPartitionsRDD[123] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:48:56 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:56 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:56 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:56 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:56 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 65 (MapPartitionsRDD[123] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:56 INFO TaskSchedulerImpl: Adding task set 65.0 with 1 tasks resource profile 0 23/06/28 07:48:56 INFO FairSchedulableBuilder: Added task set TaskSet_65.0 tasks to pool 7670323585580043917 23/06/28 07:48:56 INFO TaskSetManager: Starting task 0.0 in stage 65.0 (TID 1025) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_37 stored as values in memory (estimated size 502.7 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in memory (estimated size 126.7 KiB, free 750.7 GiB) 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on 10.60.24.15:40507 (size: 126.7 KiB, free: 750.7 GiB) 23/06/28 07:48:56 INFO SparkContext: Created broadcast 37 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:56 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on 10.60.24.10:41041 (size: 126.7 KiB, free: 487.7 GiB) 23/06/28 07:48:56 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 14 to 10.60.24.10:44288 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on 10.60.24.10:41041 (size: 12.2 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on 10.60.24.10:41041 (size: 14.1 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO BlockManagerInfo: Added rdd_119_0 in memory on 10.60.24.10:41041 (size: 2.3 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on 10.60.24.10:41041 (size: 4.5 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO TaskSetManager: Finished task 0.0 in stage 65.0 (TID 1025) in 640 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:57 INFO TaskSchedulerImpl: Removed TaskSet 65.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:57 INFO DAGScheduler: ResultStage 65 (collect at Checksum.scala:561) finished in 0.745 s 23/06/28 07:48:57 INFO DAGScheduler: Job 19 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:57 INFO TaskSchedulerImpl: Killing all running tasks in stage 65: Stage finished 23/06/28 07:48:57 INFO DAGScheduler: Job 19 finished: collect at Checksum.scala:561, took 0.747606 s 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:57 INFO CodeGenerator: Code generated in 19.566785 ms 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:57 INFO DAGScheduler: Registering RDD 126 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 15 23/06/28 07:48:57 INFO DAGScheduler: Got map stage job 20 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:57 INFO DAGScheduler: Final stage: ShuffleMapStage 67 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:57 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 66) 23/06/28 07:48:57 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:57 INFO DAGScheduler: Submitting ShuffleMapStage 67 (MapPartitionsRDD[126] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:57 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:57 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:57 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:57 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 67 (MapPartitionsRDD[126] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:57 INFO TaskSchedulerImpl: Adding task set 67.0 with 1 tasks resource profile 0 23/06/28 07:48:57 INFO FairSchedulableBuilder: Added task set TaskSet_67.0 tasks to pool 7670323585580043917 23/06/28 07:48:57 INFO TaskSetManager: Starting task 0.0 in stage 67.0 (TID 1026) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:57 INFO MemoryStore: Block broadcast_38 stored as values in memory (estimated size 473.1 KiB, free 750.7 GiB) 23/06/28 07:48:57 INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in memory (estimated size 120.7 KiB, free 750.7 GiB) 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on 10.60.24.15:40507 (size: 120.7 KiB, free: 750.7 GiB) 23/06/28 07:48:57 INFO SparkContext: Created broadcast 38 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on 10.60.24.10:41041 (size: 120.7 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO TaskSetManager: Finished task 0.0 in stage 67.0 (TID 1026) in 120 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:57 INFO TaskSchedulerImpl: Removed TaskSet 67.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:57 INFO DAGScheduler: ShuffleMapStage 67 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.139 s 23/06/28 07:48:57 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:57 INFO DAGScheduler: running: Set() 23/06/28 07:48:57 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:57 INFO DAGScheduler: failed: Set() 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:57 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:48:57 INFO DAGScheduler: Got job 21 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:48:57 INFO DAGScheduler: Final stage: ResultStage 70 (first at Snapshot.scala:271) 23/06/28 07:48:57 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 69) 23/06/28 07:48:57 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:57 INFO DAGScheduler: Submitting ResultStage 70 (MapPartitionsRDD[128] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:57 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:57 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:57 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:57 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 70 (MapPartitionsRDD[128] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:57 INFO TaskSchedulerImpl: Adding task set 70.0 with 1 tasks resource profile 0 23/06/28 07:48:57 INFO FairSchedulableBuilder: Added task set TaskSet_70.0 tasks to pool 7670323585580043917 23/06/28 07:48:57 INFO TaskSetManager: Starting task 0.0 in stage 70.0 (TID 1027) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:57 INFO MemoryStore: Block broadcast_39 stored as values in memory (estimated size 419.2 KiB, free 750.7 GiB) 23/06/28 07:48:57 INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in memory (estimated size 110.2 KiB, free 750.7 GiB) 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on 10.60.24.15:40507 (size: 110.2 KiB, free: 750.7 GiB) 23/06/28 07:48:57 INFO SparkContext: Created broadcast 39 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:57 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on 10.60.24.13:44035 (size: 110.2 KiB, free: 487.7 GiB) 23/06/28 07:48:57 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 15 to 10.60.24.13:58548 23/06/28 07:48:57 INFO TaskSetManager: Finished task 0.0 in stage 70.0 (TID 1027) in 165 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:57 INFO TaskSchedulerImpl: Removed TaskSet 70.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:57 INFO DAGScheduler: ResultStage 70 (first at Snapshot.scala:271) finished in 0.186 s 23/06/28 07:48:57 INFO DAGScheduler: Job 21 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:57 INFO TaskSchedulerImpl: Killing all running tasks in stage 70: Stage finished 23/06/28 07:48:57 INFO DAGScheduler: Job 21 finished: first at Snapshot.scala:271, took 0.188224 s 23/06/28 07:48:57 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:57 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=85, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,85,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938535000), checksumOpt=Some(VersionChecksum(Some(3c8f5a73-768f-45bb-8a07-76d582cbf470),75338,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@6dfa8024,[J@79d5090c)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-d26d832b-f936-48d3-a0bb-bfcbed64446c.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75338,1687937150000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687937150000000, MIN_INSERTION_TIME -> 1687937150000000, MAX_INSERTION_TIME -> 1687937150000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:48:57 INFO MapPartitionsRDD: Removing RDD 23 from persistence list 23/06/28 07:48:57 INFO BlockManager: Removing RDD 23 23/06/28 07:48:57 INFO OptimisticTransaction: [tableId=32485164,txnId=3c8f5a73] Committed delta #85 to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log 23/06/28 07:48:57 INFO ChecksumHook: Writing checksum file for table path dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log version 85 23/06/28 07:48:57 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:48:58 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.crc using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp 23/06/28 07:48:58 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp] Closing stream; size: 0 23/06/28 07:48:58 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp] Upload complete; size: 3471 23/06/28 07:48:58 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.crc] Starting rename. Issuing rename operation. 23/06/28 07:48:58 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.crc] Rename successful. 23/06/28 07:48:58 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000085.crc.fec61a63-837f-47d5-8ed4-fa12bb18f85f.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.crc 23/06/28 07:48:58 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:58 INFO DAGScheduler: Registering RDD 136 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 16 23/06/28 07:48:58 INFO DAGScheduler: Got map stage job 22 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 200 output partitions 23/06/28 07:48:58 INFO DAGScheduler: Final stage: ShuffleMapStage 73 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:58 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 72) 23/06/28 07:48:58 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:58 INFO DAGScheduler: Submitting ShuffleMapStage 73 (MapPartitionsRDD[136] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:58 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:58 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:58 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:58 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:58 INFO DAGScheduler: Submitting 200 missing tasks from ShuffleMapStage 73 (MapPartitionsRDD[136] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:48:58 INFO TaskSchedulerImpl: Adding task set 73.0 with 200 tasks resource profile 0 23/06/28 07:48:58 INFO FairSchedulableBuilder: Added task set TaskSet_73.0 tasks to pool 7670323585580043917 23/06/28 07:48:58 INFO DAGScheduler: Registering RDD 142 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 17 23/06/28 07:48:58 INFO DAGScheduler: Got map stage job 23 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:58 INFO DAGScheduler: Final stage: ShuffleMapStage 75 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:58 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 74) 23/06/28 07:48:58 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:58 INFO TaskSetManager: Starting task 0.0 in stage 73.0 (TID 1028) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 1.0 in stage 73.0 (TID 1029) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 2.0 in stage 73.0 (TID 1030) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 3.0 in stage 73.0 (TID 1031) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 4.0 in stage 73.0 (TID 1032) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 5.0 in stage 73.0 (TID 1033) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 6.0 in stage 73.0 (TID 1034) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 7.0 in stage 73.0 (TID 1035) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 8.0 in stage 73.0 (TID 1036) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 9.0 in stage 73.0 (TID 1037) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO DAGScheduler: Submitting ShuffleMapStage 75 (MapPartitionsRDD[142] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:58 INFO TaskSetManager: Starting task 10.0 in stage 73.0 (TID 1038) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 11.0 in stage 73.0 (TID 1039) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 12.0 in stage 73.0 (TID 1040) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 13.0 in stage 73.0 (TID 1041) (10.60.24.10, executor 0, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 14.0 in stage 73.0 (TID 1042) (10.60.24.13, executor 1, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 15.0 in stage 73.0 (TID 1043) (10.60.24.10, executor 0, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 16.0 in stage 73.0 (TID 1044) (10.60.24.13, executor 1, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 17.0 in stage 73.0 (TID 1045) (10.60.24.10, executor 0, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:58 INFO TaskSetManager: Starting task 18.0 in stage 73.0 (TID 1046) (10.60.24.13, executor 1, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 19.0 in stage 73.0 (TID 1047) (10.60.24.10, executor 0, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 20.0 in stage 73.0 (TID 1048) (10.60.24.13, executor 1, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 21.0 in stage 73.0 (TID 1049) (10.60.24.10, executor 0, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 22.0 in stage 73.0 (TID 1050) (10.60.24.13, executor 1, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 23.0 in stage 73.0 (TID 1051) (10.60.24.10, executor 0, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 24.0 in stage 73.0 (TID 1052) (10.60.24.13, executor 1, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 25.0 in stage 73.0 (TID 1053) (10.60.24.10, executor 0, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 26.0 in stage 73.0 (TID 1054) (10.60.24.13, executor 1, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 27.0 in stage 73.0 (TID 1055) (10.60.24.10, executor 0, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 28.0 in stage 73.0 (TID 1056) (10.60.24.13, executor 1, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 29.0 in stage 73.0 (TID 1057) (10.60.24.10, executor 0, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 30.0 in stage 73.0 (TID 1058) (10.60.24.13, executor 1, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 31.0 in stage 73.0 (TID 1059) (10.60.24.10, executor 0, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 32.0 in stage 73.0 (TID 1060) (10.60.24.13, executor 1, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 33.0 in stage 73.0 (TID 1061) (10.60.24.10, executor 0, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 34.0 in stage 73.0 (TID 1062) (10.60.24.13, executor 1, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 35.0 in stage 73.0 (TID 1063) (10.60.24.10, executor 0, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 36.0 in stage 73.0 (TID 1064) (10.60.24.13, executor 1, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 37.0 in stage 73.0 (TID 1065) (10.60.24.10, executor 0, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 38.0 in stage 73.0 (TID 1066) (10.60.24.13, executor 1, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 39.0 in stage 73.0 (TID 1067) (10.60.24.10, executor 0, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 40.0 in stage 73.0 (TID 1068) (10.60.24.13, executor 1, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 41.0 in stage 73.0 (TID 1069) (10.60.24.10, executor 0, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 42.0 in stage 73.0 (TID 1070) (10.60.24.13, executor 1, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 43.0 in stage 73.0 (TID 1071) (10.60.24.10, executor 0, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 44.0 in stage 73.0 (TID 1072) (10.60.24.13, executor 1, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 45.0 in stage 73.0 (TID 1073) (10.60.24.10, executor 0, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 46.0 in stage 73.0 (TID 1074) (10.60.24.13, executor 1, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 47.0 in stage 73.0 (TID 1075) (10.60.24.10, executor 0, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 48.0 in stage 73.0 (TID 1076) (10.60.24.13, executor 1, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 49.0 in stage 73.0 (TID 1077) (10.60.24.10, executor 0, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 50.0 in stage 73.0 (TID 1078) (10.60.24.13, executor 1, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 51.0 in stage 73.0 (TID 1079) (10.60.24.10, executor 0, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 52.0 in stage 73.0 (TID 1080) (10.60.24.13, executor 1, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 53.0 in stage 73.0 (TID 1081) (10.60.24.10, executor 0, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 54.0 in stage 73.0 (TID 1082) (10.60.24.13, executor 1, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 55.0 in stage 73.0 (TID 1083) (10.60.24.10, executor 0, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 56.0 in stage 73.0 (TID 1084) (10.60.24.13, executor 1, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 57.0 in stage 73.0 (TID 1085) (10.60.24.10, executor 0, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 58.0 in stage 73.0 (TID 1086) (10.60.24.13, executor 1, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 59.0 in stage 73.0 (TID 1087) (10.60.24.10, executor 0, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 60.0 in stage 73.0 (TID 1088) (10.60.24.13, executor 1, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 61.0 in stage 73.0 (TID 1089) (10.60.24.10, executor 0, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 62.0 in stage 73.0 (TID 1090) (10.60.24.13, executor 1, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 63.0 in stage 73.0 (TID 1091) (10.60.24.10, executor 0, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 64.0 in stage 73.0 (TID 1092) (10.60.24.13, executor 1, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 65.0 in stage 73.0 (TID 1093) (10.60.24.10, executor 0, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 66.0 in stage 73.0 (TID 1094) (10.60.24.13, executor 1, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 67.0 in stage 73.0 (TID 1095) (10.60.24.10, executor 0, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 68.0 in stage 73.0 (TID 1096) (10.60.24.13, executor 1, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 69.0 in stage 73.0 (TID 1097) (10.60.24.10, executor 0, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 70.0 in stage 73.0 (TID 1098) (10.60.24.13, executor 1, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 71.0 in stage 73.0 (TID 1099) (10.60.24.10, executor 0, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 72.0 in stage 73.0 (TID 1100) (10.60.24.13, executor 1, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 73.0 in stage 73.0 (TID 1101) (10.60.24.10, executor 0, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 74.0 in stage 73.0 (TID 1102) (10.60.24.13, executor 1, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 75.0 in stage 73.0 (TID 1103) (10.60.24.10, executor 0, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 76.0 in stage 73.0 (TID 1104) (10.60.24.13, executor 1, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 77.0 in stage 73.0 (TID 1105) (10.60.24.10, executor 0, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 78.0 in stage 73.0 (TID 1106) (10.60.24.13, executor 1, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 79.0 in stage 73.0 (TID 1107) (10.60.24.10, executor 0, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 80.0 in stage 73.0 (TID 1108) (10.60.24.13, executor 1, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 81.0 in stage 73.0 (TID 1109) (10.60.24.10, executor 0, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 82.0 in stage 73.0 (TID 1110) (10.60.24.13, executor 1, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 83.0 in stage 73.0 (TID 1111) (10.60.24.10, executor 0, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 84.0 in stage 73.0 (TID 1112) (10.60.24.13, executor 1, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 85.0 in stage 73.0 (TID 1113) (10.60.24.10, executor 0, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 86.0 in stage 73.0 (TID 1114) (10.60.24.13, executor 1, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 87.0 in stage 73.0 (TID 1115) (10.60.24.10, executor 0, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 88.0 in stage 73.0 (TID 1116) (10.60.24.13, executor 1, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 89.0 in stage 73.0 (TID 1117) (10.60.24.10, executor 0, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 90.0 in stage 73.0 (TID 1118) (10.60.24.13, executor 1, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 91.0 in stage 73.0 (TID 1119) (10.60.24.10, executor 0, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 92.0 in stage 73.0 (TID 1120) (10.60.24.13, executor 1, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 93.0 in stage 73.0 (TID 1121) (10.60.24.10, executor 0, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 94.0 in stage 73.0 (TID 1122) (10.60.24.13, executor 1, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 95.0 in stage 73.0 (TID 1123) (10.60.24.10, executor 0, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 96.0 in stage 73.0 (TID 1124) (10.60.24.13, executor 1, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 97.0 in stage 73.0 (TID 1125) (10.60.24.10, executor 0, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 98.0 in stage 73.0 (TID 1126) (10.60.24.13, executor 1, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 99.0 in stage 73.0 (TID 1127) (10.60.24.10, executor 0, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 100.0 in stage 73.0 (TID 1128) (10.60.24.13, executor 1, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 101.0 in stage 73.0 (TID 1129) (10.60.24.10, executor 0, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 102.0 in stage 73.0 (TID 1130) (10.60.24.13, executor 1, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 103.0 in stage 73.0 (TID 1131) (10.60.24.10, executor 0, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 104.0 in stage 73.0 (TID 1132) (10.60.24.13, executor 1, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 105.0 in stage 73.0 (TID 1133) (10.60.24.10, executor 0, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 106.0 in stage 73.0 (TID 1134) (10.60.24.13, executor 1, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 107.0 in stage 73.0 (TID 1135) (10.60.24.10, executor 0, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 108.0 in stage 73.0 (TID 1136) (10.60.24.13, executor 1, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 109.0 in stage 73.0 (TID 1137) (10.60.24.10, executor 0, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 110.0 in stage 73.0 (TID 1138) (10.60.24.13, executor 1, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 111.0 in stage 73.0 (TID 1139) (10.60.24.10, executor 0, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 112.0 in stage 73.0 (TID 1140) (10.60.24.13, executor 1, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 113.0 in stage 73.0 (TID 1141) (10.60.24.10, executor 0, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 114.0 in stage 73.0 (TID 1142) (10.60.24.13, executor 1, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 115.0 in stage 73.0 (TID 1143) (10.60.24.10, executor 0, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 116.0 in stage 73.0 (TID 1144) (10.60.24.13, executor 1, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 117.0 in stage 73.0 (TID 1145) (10.60.24.10, executor 0, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 118.0 in stage 73.0 (TID 1146) (10.60.24.13, executor 1, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 119.0 in stage 73.0 (TID 1147) (10.60.24.10, executor 0, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 120.0 in stage 73.0 (TID 1148) (10.60.24.13, executor 1, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 121.0 in stage 73.0 (TID 1149) (10.60.24.10, executor 0, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 122.0 in stage 73.0 (TID 1150) (10.60.24.13, executor 1, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 123.0 in stage 73.0 (TID 1151) (10.60.24.10, executor 0, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 124.0 in stage 73.0 (TID 1152) (10.60.24.13, executor 1, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 125.0 in stage 73.0 (TID 1153) (10.60.24.10, executor 0, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 126.0 in stage 73.0 (TID 1154) (10.60.24.13, executor 1, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 127.0 in stage 73.0 (TID 1155) (10.60.24.10, executor 0, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 128.0 in stage 73.0 (TID 1156) (10.60.24.13, executor 1, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 129.0 in stage 73.0 (TID 1157) (10.60.24.10, executor 0, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 130.0 in stage 73.0 (TID 1158) (10.60.24.13, executor 1, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 131.0 in stage 73.0 (TID 1159) (10.60.24.10, executor 0, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 132.0 in stage 73.0 (TID 1160) (10.60.24.13, executor 1, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 133.0 in stage 73.0 (TID 1161) (10.60.24.10, executor 0, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 134.0 in stage 73.0 (TID 1162) (10.60.24.13, executor 1, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 135.0 in stage 73.0 (TID 1163) (10.60.24.10, executor 0, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 136.0 in stage 73.0 (TID 1164) (10.60.24.13, executor 1, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 137.0 in stage 73.0 (TID 1165) (10.60.24.10, executor 0, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 138.0 in stage 73.0 (TID 1166) (10.60.24.13, executor 1, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 139.0 in stage 73.0 (TID 1167) (10.60.24.10, executor 0, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 140.0 in stage 73.0 (TID 1168) (10.60.24.13, executor 1, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 141.0 in stage 73.0 (TID 1169) (10.60.24.10, executor 0, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 142.0 in stage 73.0 (TID 1170) (10.60.24.13, executor 1, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 143.0 in stage 73.0 (TID 1171) (10.60.24.10, executor 0, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 144.0 in stage 73.0 (TID 1172) (10.60.24.13, executor 1, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 145.0 in stage 73.0 (TID 1173) (10.60.24.10, executor 0, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 146.0 in stage 73.0 (TID 1174) (10.60.24.13, executor 1, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 147.0 in stage 73.0 (TID 1175) (10.60.24.10, executor 0, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 148.0 in stage 73.0 (TID 1176) (10.60.24.13, executor 1, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 149.0 in stage 73.0 (TID 1177) (10.60.24.10, executor 0, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 150.0 in stage 73.0 (TID 1178) (10.60.24.13, executor 1, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 151.0 in stage 73.0 (TID 1179) (10.60.24.10, executor 0, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 152.0 in stage 73.0 (TID 1180) (10.60.24.13, executor 1, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 153.0 in stage 73.0 (TID 1181) (10.60.24.10, executor 0, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 154.0 in stage 73.0 (TID 1182) (10.60.24.13, executor 1, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 155.0 in stage 73.0 (TID 1183) (10.60.24.10, executor 0, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 156.0 in stage 73.0 (TID 1184) (10.60.24.13, executor 1, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 157.0 in stage 73.0 (TID 1185) (10.60.24.10, executor 0, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 158.0 in stage 73.0 (TID 1186) (10.60.24.13, executor 1, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 159.0 in stage 73.0 (TID 1187) (10.60.24.10, executor 0, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 160.0 in stage 73.0 (TID 1188) (10.60.24.13, executor 1, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 161.0 in stage 73.0 (TID 1189) (10.60.24.10, executor 0, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 162.0 in stage 73.0 (TID 1190) (10.60.24.13, executor 1, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 163.0 in stage 73.0 (TID 1191) (10.60.24.10, executor 0, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 164.0 in stage 73.0 (TID 1192) (10.60.24.13, executor 1, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 165.0 in stage 73.0 (TID 1193) (10.60.24.10, executor 0, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 166.0 in stage 73.0 (TID 1194) (10.60.24.13, executor 1, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 167.0 in stage 73.0 (TID 1195) (10.60.24.10, executor 0, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 168.0 in stage 73.0 (TID 1196) (10.60.24.13, executor 1, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 169.0 in stage 73.0 (TID 1197) (10.60.24.10, executor 0, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 170.0 in stage 73.0 (TID 1198) (10.60.24.13, executor 1, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 171.0 in stage 73.0 (TID 1199) (10.60.24.10, executor 0, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 172.0 in stage 73.0 (TID 1200) (10.60.24.13, executor 1, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 173.0 in stage 73.0 (TID 1201) (10.60.24.10, executor 0, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 174.0 in stage 73.0 (TID 1202) (10.60.24.13, executor 1, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 175.0 in stage 73.0 (TID 1203) (10.60.24.10, executor 0, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 176.0 in stage 73.0 (TID 1204) (10.60.24.13, executor 1, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 177.0 in stage 73.0 (TID 1205) (10.60.24.10, executor 0, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 178.0 in stage 73.0 (TID 1206) (10.60.24.13, executor 1, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 179.0 in stage 73.0 (TID 1207) (10.60.24.10, executor 0, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 180.0 in stage 73.0 (TID 1208) (10.60.24.13, executor 1, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 181.0 in stage 73.0 (TID 1209) (10.60.24.10, executor 0, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 182.0 in stage 73.0 (TID 1210) (10.60.24.13, executor 1, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 183.0 in stage 73.0 (TID 1211) (10.60.24.10, executor 0, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 184.0 in stage 73.0 (TID 1212) (10.60.24.13, executor 1, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 185.0 in stage 73.0 (TID 1213) (10.60.24.10, executor 0, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 186.0 in stage 73.0 (TID 1214) (10.60.24.13, executor 1, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 187.0 in stage 73.0 (TID 1215) (10.60.24.10, executor 0, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 188.0 in stage 73.0 (TID 1216) (10.60.24.13, executor 1, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 189.0 in stage 73.0 (TID 1217) (10.60.24.10, executor 0, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 190.0 in stage 73.0 (TID 1218) (10.60.24.13, executor 1, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 191.0 in stage 73.0 (TID 1219) (10.60.24.10, executor 0, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO MemoryStore: Block broadcast_40 stored as values in memory (estimated size 190.5 KiB, free 750.7 GiB) 23/06/28 07:48:58 INFO MemoryStore: Block broadcast_40_piece0 stored as bytes in memory (estimated size 48.0 KiB, free 750.7 GiB) 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on 10.60.24.15:40507 (size: 48.0 KiB, free: 750.7 GiB) 23/06/28 07:48:58 INFO SparkContext: Created broadcast 40 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:58 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:58 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:58 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:58 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 75 (MapPartitionsRDD[142] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:58 INFO TaskSchedulerImpl: Adding task set 75.0 with 1 tasks resource profile 0 23/06/28 07:48:58 INFO FairSchedulableBuilder: Added task set TaskSet_75.0 tasks to pool 7670323585580043917 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on 10.60.24.13:44035 (size: 48.0 KiB, free: 487.7 GiB) 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on 10.60.24.10:41041 (size: 48.0 KiB, free: 487.7 GiB) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 192.0 in stage 73.0 (TID 1220) (10.60.24.10, executor 0, partition 192, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 157.0 in stage 73.0 (TID 1185) in 250 ms on 10.60.24.10 (executor 0) (1/200) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 193.0 in stage 73.0 (TID 1221) (10.60.24.10, executor 0, partition 193, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 194.0 in stage 73.0 (TID 1222) (10.60.24.10, executor 0, partition 194, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 195.0 in stage 73.0 (TID 1223) (10.60.24.10, executor 0, partition 195, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 115.0 in stage 73.0 (TID 1143) in 251 ms on 10.60.24.10 (executor 0) (2/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 169.0 in stage 73.0 (TID 1197) in 251 ms on 10.60.24.10 (executor 0) (3/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 167.0 in stage 73.0 (TID 1195) in 251 ms on 10.60.24.10 (executor 0) (4/200) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 196.0 in stage 73.0 (TID 1224) (10.60.24.10, executor 0, partition 196, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 19.0 in stage 73.0 (TID 1047) in 253 ms on 10.60.24.10 (executor 0) (5/200) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 197.0 in stage 73.0 (TID 1225) (10.60.24.10, executor 0, partition 197, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 198.0 in stage 73.0 (TID 1226) (10.60.24.10, executor 0, partition 198, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 191.0 in stage 73.0 (TID 1219) in 251 ms on 10.60.24.10 (executor 0) (6/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 51.0 in stage 73.0 (TID 1079) in 252 ms on 10.60.24.10 (executor 0) (7/200) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 199.0 in stage 73.0 (TID 1227) (10.60.24.10, executor 0, partition 199, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 165.0 in stage 73.0 (TID 1193) in 251 ms on 10.60.24.10 (executor 0) (8/200) 23/06/28 07:48:58 INFO TaskSetManager: Starting task 0.0 in stage 75.0 (TID 1228) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 185.0 in stage 73.0 (TID 1213) in 252 ms on 10.60.24.10 (executor 0) (9/200) 23/06/28 07:48:58 INFO MemoryStore: Block broadcast_41 stored as values in memory (estimated size 471.4 KiB, free 750.6 GiB) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 127.0 in stage 73.0 (TID 1155) in 253 ms on 10.60.24.10 (executor 0) (10/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 7.0 in stage 73.0 (TID 1035) in 255 ms on 10.60.24.10 (executor 0) (11/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 47.0 in stage 73.0 (TID 1075) in 255 ms on 10.60.24.10 (executor 0) (12/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 53.0 in stage 73.0 (TID 1081) in 254 ms on 10.60.24.10 (executor 0) (13/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 49.0 in stage 73.0 (TID 1077) in 255 ms on 10.60.24.10 (executor 0) (14/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 119.0 in stage 73.0 (TID 1147) in 253 ms on 10.60.24.10 (executor 0) (15/200) 23/06/28 07:48:58 INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in memory (estimated size 119.3 KiB, free 750.6 GiB) 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on 10.60.24.15:40507 (size: 119.3 KiB, free: 750.7 GiB) 23/06/28 07:48:58 INFO SparkContext: Created broadcast 41 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:58 INFO TaskSetManager: Finished task 75.0 in stage 73.0 (TID 1103) in 269 ms on 10.60.24.10 (executor 0) (16/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 129.0 in stage 73.0 (TID 1157) in 270 ms on 10.60.24.10 (executor 0) (17/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 69.0 in stage 73.0 (TID 1097) in 271 ms on 10.60.24.10 (executor 0) (18/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 99.0 in stage 73.0 (TID 1127) in 271 ms on 10.60.24.10 (executor 0) (19/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 163.0 in stage 73.0 (TID 1191) in 270 ms on 10.60.24.10 (executor 0) (20/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 133.0 in stage 73.0 (TID 1161) in 270 ms on 10.60.24.10 (executor 0) (21/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 77.0 in stage 73.0 (TID 1105) in 277 ms on 10.60.24.10 (executor 0) (22/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 81.0 in stage 73.0 (TID 1109) in 277 ms on 10.60.24.10 (executor 0) (23/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 179.0 in stage 73.0 (TID 1207) in 277 ms on 10.60.24.10 (executor 0) (24/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 112.0 in stage 73.0 (TID 1140) in 288 ms on 10.60.24.13 (executor 1) (25/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 37.0 in stage 73.0 (TID 1065) in 295 ms on 10.60.24.10 (executor 0) (26/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 55.0 in stage 73.0 (TID 1083) in 294 ms on 10.60.24.10 (executor 0) (27/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 105.0 in stage 73.0 (TID 1133) in 304 ms on 10.60.24.10 (executor 0) (28/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 43.0 in stage 73.0 (TID 1071) in 306 ms on 10.60.24.10 (executor 0) (29/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 85.0 in stage 73.0 (TID 1113) in 305 ms on 10.60.24.10 (executor 0) (30/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 139.0 in stage 73.0 (TID 1167) in 304 ms on 10.60.24.10 (executor 0) (31/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 5.0 in stage 73.0 (TID 1033) in 306 ms on 10.60.24.10 (executor 0) (32/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 141.0 in stage 73.0 (TID 1169) in 304 ms on 10.60.24.10 (executor 0) (33/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 3.0 in stage 73.0 (TID 1031) in 306 ms on 10.60.24.10 (executor 0) (34/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 111.0 in stage 73.0 (TID 1139) in 305 ms on 10.60.24.10 (executor 0) (35/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 107.0 in stage 73.0 (TID 1135) in 306 ms on 10.60.24.10 (executor 0) (36/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 11.0 in stage 73.0 (TID 1039) in 307 ms on 10.60.24.10 (executor 0) (37/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 126.0 in stage 73.0 (TID 1154) in 312 ms on 10.60.24.13 (executor 1) (38/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 137.0 in stage 73.0 (TID 1165) in 317 ms on 10.60.24.10 (executor 0) (39/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 159.0 in stage 73.0 (TID 1187) in 317 ms on 10.60.24.10 (executor 0) (40/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 125.0 in stage 73.0 (TID 1153) in 317 ms on 10.60.24.10 (executor 0) (41/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 153.0 in stage 73.0 (TID 1181) in 317 ms on 10.60.24.10 (executor 0) (42/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 146.0 in stage 73.0 (TID 1174) in 324 ms on 10.60.24.13 (executor 1) (43/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 74.0 in stage 73.0 (TID 1102) in 339 ms on 10.60.24.13 (executor 1) (44/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 188.0 in stage 73.0 (TID 1216) in 341 ms on 10.60.24.13 (executor 1) (45/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 144.0 in stage 73.0 (TID 1172) in 341 ms on 10.60.24.13 (executor 1) (46/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 180.0 in stage 73.0 (TID 1208) in 341 ms on 10.60.24.13 (executor 1) (47/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 102.0 in stage 73.0 (TID 1130) in 354 ms on 10.60.24.13 (executor 1) (48/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 148.0 in stage 73.0 (TID 1176) in 374 ms on 10.60.24.13 (executor 1) (49/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 162.0 in stage 73.0 (TID 1190) in 375 ms on 10.60.24.13 (executor 1) (50/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 154.0 in stage 73.0 (TID 1182) in 375 ms on 10.60.24.13 (executor 1) (51/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 52.0 in stage 73.0 (TID 1080) in 389 ms on 10.60.24.13 (executor 1) (52/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 58.0 in stage 73.0 (TID 1086) in 400 ms on 10.60.24.13 (executor 1) (53/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 158.0 in stage 73.0 (TID 1186) in 399 ms on 10.60.24.13 (executor 1) (54/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 172.0 in stage 73.0 (TID 1200) in 405 ms on 10.60.24.13 (executor 1) (55/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 184.0 in stage 73.0 (TID 1212) in 405 ms on 10.60.24.13 (executor 1) (56/200) 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on 10.60.24.10:41041 (size: 119.3 KiB, free: 487.7 GiB) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 132.0 in stage 73.0 (TID 1160) in 433 ms on 10.60.24.13 (executor 1) (57/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 174.0 in stage 73.0 (TID 1202) in 433 ms on 10.60.24.13 (executor 1) (58/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 136.0 in stage 73.0 (TID 1164) in 433 ms on 10.60.24.13 (executor 1) (59/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 98.0 in stage 73.0 (TID 1126) in 436 ms on 10.60.24.13 (executor 1) (60/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 186.0 in stage 73.0 (TID 1214) in 438 ms on 10.60.24.13 (executor 1) (61/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 84.0 in stage 73.0 (TID 1112) in 444 ms on 10.60.24.13 (executor 1) (62/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 138.0 in stage 73.0 (TID 1166) in 468 ms on 10.60.24.13 (executor 1) (63/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 166.0 in stage 73.0 (TID 1194) in 472 ms on 10.60.24.13 (executor 1) (64/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 190.0 in stage 73.0 (TID 1218) in 472 ms on 10.60.24.13 (executor 1) (65/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 122.0 in stage 73.0 (TID 1150) in 488 ms on 10.60.24.13 (executor 1) (66/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 150.0 in stage 73.0 (TID 1178) in 506 ms on 10.60.24.13 (executor 1) (67/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 164.0 in stage 73.0 (TID 1192) in 506 ms on 10.60.24.13 (executor 1) (68/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 104.0 in stage 73.0 (TID 1132) in 508 ms on 10.60.24.13 (executor 1) (69/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 29.0 in stage 73.0 (TID 1057) in 509 ms on 10.60.24.10 (executor 0) (70/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 82.0 in stage 73.0 (TID 1110) in 511 ms on 10.60.24.13 (executor 1) (71/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 18.0 in stage 73.0 (TID 1046) in 512 ms on 10.60.24.13 (executor 1) (72/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 50.0 in stage 73.0 (TID 1078) in 511 ms on 10.60.24.13 (executor 1) (73/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 152.0 in stage 73.0 (TID 1180) in 510 ms on 10.60.24.13 (executor 1) (74/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 20.0 in stage 73.0 (TID 1048) in 512 ms on 10.60.24.13 (executor 1) (75/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 134.0 in stage 73.0 (TID 1162) in 510 ms on 10.60.24.13 (executor 1) (76/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 143.0 in stage 73.0 (TID 1171) in 518 ms on 10.60.24.10 (executor 0) (77/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 170.0 in stage 73.0 (TID 1198) in 534 ms on 10.60.24.13 (executor 1) (78/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 160.0 in stage 73.0 (TID 1188) in 534 ms on 10.60.24.13 (executor 1) (79/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 120.0 in stage 73.0 (TID 1148) in 539 ms on 10.60.24.13 (executor 1) (80/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 88.0 in stage 73.0 (TID 1116) in 540 ms on 10.60.24.13 (executor 1) (81/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 95.0 in stage 73.0 (TID 1123) in 540 ms on 10.60.24.10 (executor 0) (82/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 145.0 in stage 73.0 (TID 1173) in 540 ms on 10.60.24.10 (executor 0) (83/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 171.0 in stage 73.0 (TID 1199) in 540 ms on 10.60.24.10 (executor 0) (84/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 130.0 in stage 73.0 (TID 1158) in 542 ms on 10.60.24.13 (executor 1) (85/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 189.0 in stage 73.0 (TID 1217) in 542 ms on 10.60.24.10 (executor 0) (86/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 121.0 in stage 73.0 (TID 1149) in 542 ms on 10.60.24.10 (executor 0) (87/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 57.0 in stage 73.0 (TID 1085) in 543 ms on 10.60.24.10 (executor 0) (88/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 1.0 in stage 73.0 (TID 1029) in 544 ms on 10.60.24.10 (executor 0) (89/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 76.0 in stage 73.0 (TID 1104) in 545 ms on 10.60.24.13 (executor 1) (90/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 182.0 in stage 73.0 (TID 1210) in 544 ms on 10.60.24.13 (executor 1) (91/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 83.0 in stage 73.0 (TID 1111) in 548 ms on 10.60.24.10 (executor 0) (92/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 100.0 in stage 73.0 (TID 1128) in 550 ms on 10.60.24.13 (executor 1) (93/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 80.0 in stage 73.0 (TID 1108) in 550 ms on 10.60.24.13 (executor 1) (94/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 36.0 in stage 73.0 (TID 1064) in 551 ms on 10.60.24.13 (executor 1) (95/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 114.0 in stage 73.0 (TID 1142) in 553 ms on 10.60.24.13 (executor 1) (96/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 71.0 in stage 73.0 (TID 1099) in 554 ms on 10.60.24.10 (executor 0) (97/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 91.0 in stage 73.0 (TID 1119) in 554 ms on 10.60.24.10 (executor 0) (98/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 25.0 in stage 73.0 (TID 1053) in 555 ms on 10.60.24.10 (executor 0) (99/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 187.0 in stage 73.0 (TID 1215) in 555 ms on 10.60.24.10 (executor 0) (100/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 183.0 in stage 73.0 (TID 1211) in 556 ms on 10.60.24.10 (executor 0) (101/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 64.0 in stage 73.0 (TID 1092) in 558 ms on 10.60.24.13 (executor 1) (102/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 63.0 in stage 73.0 (TID 1091) in 559 ms on 10.60.24.10 (executor 0) (103/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 24.0 in stage 73.0 (TID 1052) in 562 ms on 10.60.24.13 (executor 1) (104/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 41.0 in stage 73.0 (TID 1069) in 563 ms on 10.60.24.10 (executor 0) (105/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 44.0 in stage 73.0 (TID 1072) in 566 ms on 10.60.24.13 (executor 1) (106/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 108.0 in stage 73.0 (TID 1136) in 566 ms on 10.60.24.13 (executor 1) (107/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 42.0 in stage 73.0 (TID 1070) in 567 ms on 10.60.24.13 (executor 1) (108/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 13.0 in stage 73.0 (TID 1041) in 568 ms on 10.60.24.10 (executor 0) (109/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 9.0 in stage 73.0 (TID 1037) in 568 ms on 10.60.24.10 (executor 0) (110/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 62.0 in stage 73.0 (TID 1090) in 569 ms on 10.60.24.13 (executor 1) (111/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 8.0 in stage 73.0 (TID 1036) in 570 ms on 10.60.24.13 (executor 1) (112/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 4.0 in stage 73.0 (TID 1032) in 570 ms on 10.60.24.13 (executor 1) (113/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 176.0 in stage 73.0 (TID 1204) in 570 ms on 10.60.24.13 (executor 1) (114/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 106.0 in stage 73.0 (TID 1134) in 574 ms on 10.60.24.13 (executor 1) (115/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 123.0 in stage 73.0 (TID 1151) in 574 ms on 10.60.24.10 (executor 0) (116/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 175.0 in stage 73.0 (TID 1203) in 574 ms on 10.60.24.10 (executor 0) (117/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 39.0 in stage 73.0 (TID 1067) in 576 ms on 10.60.24.10 (executor 0) (118/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 161.0 in stage 73.0 (TID 1189) in 575 ms on 10.60.24.10 (executor 0) (119/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 135.0 in stage 73.0 (TID 1163) in 576 ms on 10.60.24.10 (executor 0) (120/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 72.0 in stage 73.0 (TID 1100) in 577 ms on 10.60.24.13 (executor 1) (121/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 23.0 in stage 73.0 (TID 1051) in 578 ms on 10.60.24.10 (executor 0) (122/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 140.0 in stage 73.0 (TID 1168) in 576 ms on 10.60.24.13 (executor 1) (123/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 194.0 in stage 73.0 (TID 1222) in 326 ms on 10.60.24.10 (executor 0) (124/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 142.0 in stage 73.0 (TID 1170) in 576 ms on 10.60.24.13 (executor 1) (125/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 17.0 in stage 73.0 (TID 1045) in 578 ms on 10.60.24.10 (executor 0) (126/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 116.0 in stage 73.0 (TID 1144) in 580 ms on 10.60.24.13 (executor 1) (127/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 90.0 in stage 73.0 (TID 1118) in 581 ms on 10.60.24.13 (executor 1) (128/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 0.0 in stage 73.0 (TID 1028) in 583 ms on 10.60.24.13 (executor 1) (129/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 168.0 in stage 73.0 (TID 1196) in 580 ms on 10.60.24.13 (executor 1) (130/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 60.0 in stage 73.0 (TID 1088) in 581 ms on 10.60.24.13 (executor 1) (131/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 79.0 in stage 73.0 (TID 1107) in 582 ms on 10.60.24.10 (executor 0) (132/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 35.0 in stage 73.0 (TID 1063) in 584 ms on 10.60.24.10 (executor 0) (133/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 199.0 in stage 73.0 (TID 1227) in 331 ms on 10.60.24.10 (executor 0) (134/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 101.0 in stage 73.0 (TID 1129) in 584 ms on 10.60.24.10 (executor 0) (135/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 32.0 in stage 73.0 (TID 1060) in 588 ms on 10.60.24.13 (executor 1) (136/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 92.0 in stage 73.0 (TID 1120) in 587 ms on 10.60.24.13 (executor 1) (137/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 26.0 in stage 73.0 (TID 1054) in 588 ms on 10.60.24.13 (executor 1) (138/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 28.0 in stage 73.0 (TID 1056) in 588 ms on 10.60.24.13 (executor 1) (139/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 181.0 in stage 73.0 (TID 1209) in 587 ms on 10.60.24.10 (executor 0) (140/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 149.0 in stage 73.0 (TID 1177) in 587 ms on 10.60.24.10 (executor 0) (141/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 65.0 in stage 73.0 (TID 1093) in 588 ms on 10.60.24.10 (executor 0) (142/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 87.0 in stage 73.0 (TID 1115) in 588 ms on 10.60.24.10 (executor 0) (143/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 192.0 in stage 73.0 (TID 1220) in 337 ms on 10.60.24.10 (executor 0) (144/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 198.0 in stage 73.0 (TID 1226) in 337 ms on 10.60.24.10 (executor 0) (145/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 173.0 in stage 73.0 (TID 1201) in 588 ms on 10.60.24.10 (executor 0) (146/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 124.0 in stage 73.0 (TID 1152) in 588 ms on 10.60.24.13 (executor 1) (147/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 96.0 in stage 73.0 (TID 1124) in 590 ms on 10.60.24.13 (executor 1) (148/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 40.0 in stage 73.0 (TID 1068) in 591 ms on 10.60.24.13 (executor 1) (149/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 86.0 in stage 73.0 (TID 1114) in 590 ms on 10.60.24.13 (executor 1) (150/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 156.0 in stage 73.0 (TID 1184) in 589 ms on 10.60.24.13 (executor 1) (151/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 66.0 in stage 73.0 (TID 1094) in 591 ms on 10.60.24.13 (executor 1) (152/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 34.0 in stage 73.0 (TID 1062) in 592 ms on 10.60.24.13 (executor 1) (153/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 14.0 in stage 73.0 (TID 1042) in 592 ms on 10.60.24.13 (executor 1) (154/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 10.0 in stage 73.0 (TID 1038) in 592 ms on 10.60.24.13 (executor 1) (155/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 30.0 in stage 73.0 (TID 1058) in 592 ms on 10.60.24.13 (executor 1) (156/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 22.0 in stage 73.0 (TID 1050) in 592 ms on 10.60.24.13 (executor 1) (157/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 16.0 in stage 73.0 (TID 1044) in 592 ms on 10.60.24.13 (executor 1) (158/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 48.0 in stage 73.0 (TID 1076) in 592 ms on 10.60.24.13 (executor 1) (159/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 38.0 in stage 73.0 (TID 1066) in 592 ms on 10.60.24.13 (executor 1) (160/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 193.0 in stage 73.0 (TID 1221) in 341 ms on 10.60.24.10 (executor 0) (161/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 177.0 in stage 73.0 (TID 1205) in 591 ms on 10.60.24.10 (executor 0) (162/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 89.0 in stage 73.0 (TID 1117) in 592 ms on 10.60.24.10 (executor 0) (163/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 97.0 in stage 73.0 (TID 1125) in 593 ms on 10.60.24.10 (executor 0) (164/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 31.0 in stage 73.0 (TID 1059) in 594 ms on 10.60.24.10 (executor 0) (165/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 109.0 in stage 73.0 (TID 1137) in 593 ms on 10.60.24.10 (executor 0) (166/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 54.0 in stage 73.0 (TID 1082) in 593 ms on 10.60.24.13 (executor 1) (167/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 12.0 in stage 73.0 (TID 1040) in 594 ms on 10.60.24.13 (executor 1) (168/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 27.0 in stage 73.0 (TID 1055) in 596 ms on 10.60.24.10 (executor 0) (169/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 103.0 in stage 73.0 (TID 1131) in 595 ms on 10.60.24.10 (executor 0) (170/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 131.0 in stage 73.0 (TID 1159) in 594 ms on 10.60.24.10 (executor 0) (171/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 113.0 in stage 73.0 (TID 1141) in 595 ms on 10.60.24.10 (executor 0) (172/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 151.0 in stage 73.0 (TID 1179) in 594 ms on 10.60.24.10 (executor 0) (173/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 195.0 in stage 73.0 (TID 1223) in 344 ms on 10.60.24.10 (executor 0) (174/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 61.0 in stage 73.0 (TID 1089) in 595 ms on 10.60.24.10 (executor 0) (175/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 15.0 in stage 73.0 (TID 1043) in 597 ms on 10.60.24.10 (executor 0) (176/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 93.0 in stage 73.0 (TID 1121) in 596 ms on 10.60.24.10 (executor 0) (177/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 59.0 in stage 73.0 (TID 1087) in 596 ms on 10.60.24.10 (executor 0) (178/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 155.0 in stage 73.0 (TID 1183) in 595 ms on 10.60.24.10 (executor 0) (179/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 110.0 in stage 73.0 (TID 1138) in 596 ms on 10.60.24.13 (executor 1) (180/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 196.0 in stage 73.0 (TID 1224) in 344 ms on 10.60.24.10 (executor 0) (181/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 46.0 in stage 73.0 (TID 1074) in 597 ms on 10.60.24.13 (executor 1) (182/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 117.0 in stage 73.0 (TID 1145) in 595 ms on 10.60.24.10 (executor 0) (183/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 118.0 in stage 73.0 (TID 1146) in 595 ms on 10.60.24.13 (executor 1) (184/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 147.0 in stage 73.0 (TID 1175) in 595 ms on 10.60.24.10 (executor 0) (185/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 6.0 in stage 73.0 (TID 1034) in 598 ms on 10.60.24.13 (executor 1) (186/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 73.0 in stage 73.0 (TID 1101) in 597 ms on 10.60.24.10 (executor 0) (187/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 21.0 in stage 73.0 (TID 1049) in 598 ms on 10.60.24.10 (executor 0) (188/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 2.0 in stage 73.0 (TID 1030) in 598 ms on 10.60.24.13 (executor 1) (189/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 70.0 in stage 73.0 (TID 1098) in 597 ms on 10.60.24.13 (executor 1) (190/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 67.0 in stage 73.0 (TID 1095) in 597 ms on 10.60.24.10 (executor 0) (191/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 68.0 in stage 73.0 (TID 1096) in 598 ms on 10.60.24.13 (executor 1) (192/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 128.0 in stage 73.0 (TID 1156) in 597 ms on 10.60.24.13 (executor 1) (193/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 33.0 in stage 73.0 (TID 1061) in 602 ms on 10.60.24.10 (executor 0) (194/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 45.0 in stage 73.0 (TID 1073) in 602 ms on 10.60.24.10 (executor 0) (195/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 178.0 in stage 73.0 (TID 1206) in 607 ms on 10.60.24.13 (executor 1) (196/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 56.0 in stage 73.0 (TID 1084) in 608 ms on 10.60.24.13 (executor 1) (197/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 94.0 in stage 73.0 (TID 1122) in 608 ms on 10.60.24.13 (executor 1) (198/200) 23/06/28 07:48:58 INFO TaskSetManager: Finished task 78.0 in stage 73.0 (TID 1106) in 612 ms on 10.60.24.13 (executor 1) (199/200) 23/06/28 07:48:58 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:48:58 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to 10.60.24.10:44288 23/06/28 07:48:58 INFO TaskSetManager: Finished task 197.0 in stage 73.0 (TID 1225) in 374 ms on 10.60.24.10 (executor 0) (200/200) 23/06/28 07:48:58 INFO TaskSchedulerImpl: Removed TaskSet 73.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:58 INFO DAGScheduler: ShuffleMapStage 73 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.632 s 23/06/28 07:48:58 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:58 INFO DAGScheduler: running: Set(ShuffleMapStage 75) 23/06/28 07:48:58 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:58 INFO DAGScheduler: failed: Set() 23/06/28 07:48:58 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on 10.60.24.10:41041 (size: 12.2 KiB, free: 487.7 GiB) 23/06/28 07:48:58 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on 10.60.24.10:41041 (size: 1650.0 B, free: 487.7 GiB) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 0.0 in stage 75.0 (TID 1228) in 463 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Removed TaskSet 75.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:59 INFO DAGScheduler: ShuffleMapStage 75 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.717 s 23/06/28 07:48:59 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:59 INFO DAGScheduler: running: Set() 23/06/28 07:48:59 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:59 INFO DAGScheduler: failed: Set() 23/06/28 07:48:59 INFO ShufflePartitionsUtil: For shuffle(17), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:59 INFO DAGScheduler: Registering RDD 144 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 18 23/06/28 07:48:59 INFO DAGScheduler: Got map stage job 24 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:48:59 INFO DAGScheduler: Final stage: ShuffleMapStage 78 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:59 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 77) 23/06/28 07:48:59 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:59 INFO DAGScheduler: Submitting ShuffleMapStage 78 (MapPartitionsRDD[144] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:59 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:59 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:59 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:59 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 78 (MapPartitionsRDD[144] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Adding task set 78.0 with 1 tasks resource profile 0 23/06/28 07:48:59 INFO FairSchedulableBuilder: Added task set TaskSet_78.0 tasks to pool 7670323585580043917 23/06/28 07:48:59 INFO TaskSetManager: Starting task 0.0 in stage 78.0 (TID 1229) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_42 stored as values in memory (estimated size 127.6 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_42_piece0 stored as bytes in memory (estimated size 20.5 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on 10.60.24.15:40507 (size: 20.5 KiB, free: 750.7 GiB) 23/06/28 07:48:59 INFO SparkContext: Created broadcast 42 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on 10.60.24.10:41041 (size: 20.5 KiB, free: 487.7 GiB) 23/06/28 07:48:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 17 to 10.60.24.10:44288 23/06/28 07:48:59 INFO TaskSetManager: Finished task 0.0 in stage 78.0 (TID 1229) in 22 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Removed TaskSet 78.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:59 INFO DAGScheduler: ShuffleMapStage 78 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.025 s 23/06/28 07:48:59 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:59 INFO DAGScheduler: running: Set() 23/06/28 07:48:59 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:59 INFO DAGScheduler: failed: Set() 23/06/28 07:48:59 INFO ShufflePartitionsUtil: For shuffle(16), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:48:59 INFO HashAggregateExec: spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current version of codegened fast hashmap does not support this aggregate. 23/06/28 07:48:59 INFO SparkContext: Starting job: collectResult at VacuumCommandImplEdge.scala:251 23/06/28 07:48:59 INFO DAGScheduler: Got job 25 (collectResult at VacuumCommandImplEdge.scala:251) with 1 output partitions 23/06/28 07:48:59 INFO DAGScheduler: Final stage: ResultStage 85 (collectResult at VacuumCommandImplEdge.scala:251) 23/06/28 07:48:59 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 84, ShuffleMapStage 81) 23/06/28 07:48:59 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:59 INFO DAGScheduler: Submitting ResultStage 85 (MapPartitionsRDD[154] at collectResult at VacuumCommandImplEdge.scala:251), which has no missing parents 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:59 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:59 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:59 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:59 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 85 (MapPartitionsRDD[154] at collectResult at VacuumCommandImplEdge.scala:251) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Adding task set 85.0 with 1 tasks resource profile 0 23/06/28 07:48:59 INFO FairSchedulableBuilder: Added task set TaskSet_85.0 tasks to pool 7670323585580043917 23/06/28 07:48:59 INFO TaskSetManager: Starting task 0.0 in stage 85.0 (TID 1230) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_43 stored as values in memory (estimated size 471.6 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in memory (estimated size 127.6 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on 10.60.24.15:40507 (size: 127.6 KiB, free: 750.7 GiB) 23/06/28 07:48:59 INFO SparkContext: Created broadcast 43 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on 10.60.24.13:44035 (size: 127.6 KiB, free: 487.7 GiB) 23/06/28 07:48:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 16 to 10.60.24.13:58548 23/06/28 07:48:59 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 18 to 10.60.24.13:58548 23/06/28 07:48:59 INFO TaskSetManager: Finished task 0.0 in stage 85.0 (TID 1230) in 96 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Removed TaskSet 85.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:59 INFO DAGScheduler: ResultStage 85 (collectResult at VacuumCommandImplEdge.scala:251) finished in 0.120 s 23/06/28 07:48:59 INFO DAGScheduler: Job 25 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:48:59 INFO TaskSchedulerImpl: Killing all running tasks in stage 85: Stage finished 23/06/28 07:48:59 INFO DAGScheduler: Job 25 finished: collectResult at VacuumCommandImplEdge.scala:251, took 0.121702 s 23/06/28 07:48:59 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_18_370064497) 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:59 WARN BlockManager: Asked to remove block spark_join_relation_18_370064497, which does not exist 23/06/28 07:48:59 INFO AzureBlobFileSystem:V3: FS_OP_DELETE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/year=2023/month=06/day=19] Starting delete - recursive false 23/06/28 07:48:59 INFO AzureBlobFileSystem:V3: FS_OP_DELETE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/year=2023/month=06/day=19] Delete successful. 23/06/28 07:48:59 INFO OptimisticTransaction: [tableId=32485164,txnId=9cb1f817] Attempting to commit version 86 with 1 actions with SnapshotIsolation isolation level 23/06/28 07:48:59 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:48:59 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp 23/06/28 07:48:59 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp] Closing stream; size: 0 23/06/28 07:48:59 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp] Upload complete; size: 746 23/06/28 07:48:59 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json] Starting rename. Issuing rename operation. 23/06/28 07:48:59 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json] Rename successful. 23/06/28 07:48:59 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.json.fc1ae0ef-2a86-4d70-9973-0ccc6f79cc87.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json 23/06/28 07:48:59 INFO OptimisticTransaction: [tableId=32485164,txnId=9cb1f817] Incremental commit: starting with snapshot version 85 23/06/28 07:48:59 INFO DeltaLog: Loading version 86 starting from checkpoint version 80. 23/06/28 07:48:59 INFO SnapshotEdge: [tableId=32485164-a88a-4b48-ac48-c8898e21a3ee] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=86, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,86,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687938539000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC22229052"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938539000), checksumOpt=Some(VersionChecksum(Some(9cb1f817-afcb-40db-a76f-91f4189ae557),75338,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@59366b2f,[J@5ee48ba)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-d26d832b-f936-48d3-a0bb-bfcbed64446c.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75338,1687937150000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687937150000000, MIN_INSERTION_TIME -> 1687937150000000, MAX_INSERTION_TIME -> 1687937150000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_44 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_44_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:48:59 INFO SparkContext: Created broadcast 44 from broadcast at DeltaLog.scala:634 23/06/28 07:48:59 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 27842) 23/06/28 07:48:59 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 6, totalFileSize: 7282) 23/06/28 07:48:59 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:48:59 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_45 stored as values in memory (estimated size 609.9 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_45_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:48:59 INFO SparkContext: Created broadcast 45 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:48:59 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:48:59 INFO DAGScheduler: Registering RDD 161 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 19 23/06/28 07:48:59 INFO DAGScheduler: Got map stage job 26 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 11 output partitions 23/06/28 07:48:59 INFO DAGScheduler: Final stage: ShuffleMapStage 86 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:48:59 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:48:59 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:59 INFO DAGScheduler: Submitting ShuffleMapStage 86 (MapPartitionsRDD[161] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:48:59 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:59 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:59 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:59 INFO DAGScheduler: Submitting 11 missing tasks from ShuffleMapStage 86 (MapPartitionsRDD[161] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Adding task set 86.0 with 11 tasks resource profile 0 23/06/28 07:48:59 INFO FairSchedulableBuilder: Added task set TaskSet_86.0 tasks to pool 7670323585580043917 23/06/28 07:48:59 INFO TaskSetManager: Starting task 0.0 in stage 86.0 (TID 1231) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 1.0 in stage 86.0 (TID 1232) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 2.0 in stage 86.0 (TID 1233) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 3.0 in stage 86.0 (TID 1234) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 4.0 in stage 86.0 (TID 1235) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 5.0 in stage 86.0 (TID 1236) (10.60.24.13, executor 1, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 6.0 in stage 86.0 (TID 1237) (10.60.24.10, executor 0, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 7.0 in stage 86.0 (TID 1238) (10.60.24.13, executor 1, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 8.0 in stage 86.0 (TID 1239) (10.60.24.10, executor 0, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 9.0 in stage 86.0 (TID 1240) (10.60.24.13, executor 1, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO TaskSetManager: Starting task 10.0 in stage 86.0 (TID 1241) (10.60.24.10, executor 0, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_46 stored as values in memory (estimated size 324.3 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in memory (estimated size 79.3 KiB, free 750.6 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on 10.60.24.15:40507 (size: 79.3 KiB, free: 750.7 GiB) 23/06/28 07:48:59 INFO SparkContext: Created broadcast 46 from broadcast at TaskSetManager.scala:627 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on 10.60.24.10:41041 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on 10.60.24.13:44035 (size: 79.3 KiB, free: 487.7 GiB) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 6.0 in stage 86.0 (TID 1237) in 18 ms on 10.60.24.10 (executor 0) (1/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 10.0 in stage 86.0 (TID 1241) in 24 ms on 10.60.24.10 (executor 0) (2/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 2.0 in stage 86.0 (TID 1233) in 24 ms on 10.60.24.10 (executor 0) (3/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 8.0 in stage 86.0 (TID 1239) in 25 ms on 10.60.24.10 (executor 0) (4/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 7.0 in stage 86.0 (TID 1238) in 26 ms on 10.60.24.13 (executor 1) (5/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 1.0 in stage 86.0 (TID 1232) in 26 ms on 10.60.24.13 (executor 1) (6/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 3.0 in stage 86.0 (TID 1234) in 26 ms on 10.60.24.13 (executor 1) (7/11) 23/06/28 07:48:59 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 4.0 in stage 86.0 (TID 1235) in 26 ms on 10.60.24.10 (executor 0) (8/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 9.0 in stage 86.0 (TID 1240) in 26 ms on 10.60.24.13 (executor 1) (9/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 5.0 in stage 86.0 (TID 1236) in 26 ms on 10.60.24.13 (executor 1) (10/11) 23/06/28 07:48:59 INFO TaskSetManager: Finished task 0.0 in stage 86.0 (TID 1231) in 63 ms on 10.60.24.10 (executor 0) (11/11) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Removed TaskSet 86.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:48:59 INFO DAGScheduler: ShuffleMapStage 86 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.075 s 23/06/28 07:48:59 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:48:59 INFO DAGScheduler: running: Set() 23/06/28 07:48:59 INFO DAGScheduler: waiting: Set() 23/06/28 07:48:59 INFO DAGScheduler: failed: Set() 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:48:59 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:48:59 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":5103,"lifetime":238371},"10.60.24.13":{"diskUsage":0,"lifetime":238641}}, ioCacheNumScanTasks={"numLocalScanTasks":3,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:48:59 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:48:59 INFO DAGScheduler: Got job 27 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:48:59 INFO DAGScheduler: Final stage: ResultStage 88 (collect at Checksum.scala:561) 23/06/28 07:48:59 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 87) 23/06/28 07:48:59 INFO DAGScheduler: Missing parents: List() 23/06/28 07:48:59 INFO DAGScheduler: Submitting ResultStage 88 (MapPartitionsRDD[172] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:48:59 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:48:59 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:48:59 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:48:59 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 88 (MapPartitionsRDD[172] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:48:59 INFO TaskSchedulerImpl: Adding task set 88.0 with 1 tasks resource profile 0 23/06/28 07:48:59 INFO FairSchedulableBuilder: Added task set TaskSet_88.0 tasks to pool 7670323585580043917 23/06/28 07:48:59 INFO TaskSetManager: Starting task 0.0 in stage 88.0 (TID 1242) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:48:59 INFO MemoryStore: Block broadcast_47 stored as values in memory (estimated size 502.7 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in memory (estimated size 126.6 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on 10.60.24.15:40507 (size: 126.6 KiB, free: 750.7 GiB) 23/06/28 07:49:00 INFO SparkContext: Created broadcast 47 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on 10.60.24.10:41041 (size: 126.6 KiB, free: 487.7 GiB) 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 19 to 10.60.24.10:44288 23/06/28 07:49:00 INFO BlockManagerInfo: Added rdd_168_0 in memory on 10.60.24.10:41041 (size: 2.3 KiB, free: 487.7 GiB) 23/06/28 07:49:00 INFO TaskSetManager: Finished task 0.0 in stage 88.0 (TID 1242) in 63 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:00 INFO TaskSchedulerImpl: Removed TaskSet 88.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:00 INFO DAGScheduler: ResultStage 88 (collect at Checksum.scala:561) finished in 0.118 s 23/06/28 07:49:00 INFO DAGScheduler: Job 27 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:00 INFO TaskSchedulerImpl: Killing all running tasks in stage 88: Stage finished 23/06/28 07:49:00 INFO DAGScheduler: Job 27 finished: collect at Checksum.scala:561, took 0.119973 s 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:49:00 INFO DAGScheduler: Registering RDD 175 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 20 23/06/28 07:49:00 INFO DAGScheduler: Got map stage job 28 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:49:00 INFO DAGScheduler: Final stage: ShuffleMapStage 90 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:49:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 89) 23/06/28 07:49:00 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:00 INFO DAGScheduler: Submitting ShuffleMapStage 90 (MapPartitionsRDD[175] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:00 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:00 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:00 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:00 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 90 (MapPartitionsRDD[175] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:00 INFO TaskSchedulerImpl: Adding task set 90.0 with 1 tasks resource profile 0 23/06/28 07:49:00 INFO FairSchedulableBuilder: Added task set TaskSet_90.0 tasks to pool 7670323585580043917 23/06/28 07:49:00 INFO TaskSetManager: Starting task 0.0 in stage 90.0 (TID 1243) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_48 stored as values in memory (estimated size 473.1 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in memory (estimated size 120.7 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on 10.60.24.15:40507 (size: 120.7 KiB, free: 750.7 GiB) 23/06/28 07:49:00 INFO SparkContext: Created broadcast 48 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on 10.60.24.10:41041 (size: 120.7 KiB, free: 487.7 GiB) 23/06/28 07:49:00 INFO TaskSetManager: Finished task 0.0 in stage 90.0 (TID 1243) in 35 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:00 INFO TaskSchedulerImpl: Removed TaskSet 90.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:00 INFO DAGScheduler: ShuffleMapStage 90 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.048 s 23/06/28 07:49:00 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:00 INFO DAGScheduler: running: Set() 23/06/28 07:49:00 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:00 INFO DAGScheduler: failed: Set() 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:49:00 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:49:00 INFO DAGScheduler: Got job 29 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:49:00 INFO DAGScheduler: Final stage: ResultStage 93 (first at Snapshot.scala:271) 23/06/28 07:49:00 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 92) 23/06/28 07:49:00 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:00 INFO DAGScheduler: Submitting ResultStage 93 (MapPartitionsRDD[177] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:00 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:00 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:00 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:00 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 93 (MapPartitionsRDD[177] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:00 INFO TaskSchedulerImpl: Adding task set 93.0 with 1 tasks resource profile 0 23/06/28 07:49:00 INFO FairSchedulableBuilder: Added task set TaskSet_93.0 tasks to pool 7670323585580043917 23/06/28 07:49:00 INFO TaskSetManager: Starting task 0.0 in stage 93.0 (TID 1244) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_49 stored as values in memory (estimated size 419.2 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_49_piece0 stored as bytes in memory (estimated size 110.2 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on 10.60.24.15:40507 (size: 110.2 KiB, free: 750.7 GiB) 23/06/28 07:49:00 INFO SparkContext: Created broadcast 49 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on 10.60.24.10:41041 (size: 110.2 KiB, free: 487.7 GiB) 23/06/28 07:49:00 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 20 to 10.60.24.10:44288 23/06/28 07:49:00 INFO TaskSetManager: Finished task 0.0 in stage 93.0 (TID 1244) in 37 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:00 INFO TaskSchedulerImpl: Removed TaskSet 93.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:00 INFO DAGScheduler: ResultStage 93 (first at Snapshot.scala:271) finished in 0.053 s 23/06/28 07:49:00 INFO DAGScheduler: Job 29 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:00 INFO TaskSchedulerImpl: Killing all running tasks in stage 93: Stage finished 23/06/28 07:49:00 INFO DAGScheduler: Job 29 finished: first at Snapshot.scala:271, took 0.054377 s 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:00 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=86, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,86,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687938539000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC22229052"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938539000), checksumOpt=Some(VersionChecksum(Some(9cb1f817-afcb-40db-a76f-91f4189ae557),75338,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@59366b2f,[J@5ee48ba)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-d26d832b-f936-48d3-a0bb-bfcbed64446c.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75338,1687937150000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:25:08.937Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687937150000000, MIN_INSERTION_TIME -> 1687937150000000, MAX_INSERTION_TIME -> 1687937150000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:49:00 INFO MapPartitionsRDD: Removing RDD 119 from persistence list 23/06/28 07:49:00 INFO BlockManager: Removing RDD 119 23/06/28 07:49:00 INFO OptimisticTransaction: [tableId=32485164,txnId=9cb1f817] Committed delta #86 to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log 23/06/28 07:49:00 INFO ChecksumHook: Writing checksum file for table path dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log version 86 23/06/28 07:49:00 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:49:00 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.crc using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp 23/06/28 07:49:00 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp] Closing stream; size: 0 23/06/28 07:49:00 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp] Upload complete; size: 3471 23/06/28 07:49:00 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.crc] Starting rename. Issuing rename operation. 23/06/28 07:49:00 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.crc] Rename successful. 23/06/28 07:49:00 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000086.crc.25076611-8848-4886-ba76-960e576678c7.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.crc 23/06/28 07:49:00 INFO MapPartitionsRDD: Removing RDD 39 from persistence list 23/06/28 07:49:00 INFO BlockManager: Removing RDD 39 23/06/28 07:49:00 INFO CodeGenerator: Code generated in 7.306456 ms 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:49:00 INFO CodeGenerator: Code generated in 4.95955 ms 23/06/28 07:49:00 INFO ClusterLoadMonitor: Removed query with execution ID:71. Current active queries:0 23/06/28 07:49:00 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:00 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:00 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:00.425Z","run":{"runId":"a30efe08-5875-4c12-b7ed-8539e506e38f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.directory.VacuumTableCommand","num-children":0,"table":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw"},"dryRun":false,"options":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_vacuum_table_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_database: raw 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO ClusterLoadMonitor: Added query with execution ID:85. Current active queries:1 23/06/28 07:49:00 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:00 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:00 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:00 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:49:00.721Z","run":{"runId":"1550b6da-2940-48d7-a295-e71bc717893b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.ShowPartitionsDeltaCommand","num-children":0,"deltaTableIdentifier":null,"snapshot":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_show_partitions_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:00 INFO SnapshotEdge: [tableId=32485164-a88a-4b48-ac48-c8898e21a3ee] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_50 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:49:00 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:49:00 INFO SparkContext: Created broadcast 50 from broadcast at DeltaLog.scala:634 23/06/28 07:49:00 INFO CodeGenerator: Code generated in 25.043476 ms 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 21.239528 ms 23/06/28 07:49:01 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:49:01 INFO DAGScheduler: Got job 30 (executeCollect at DatasetRefCache.scala:71) with 10 output partitions 23/06/28 07:49:01 INFO DAGScheduler: Final stage: ResultStage 94 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:49:01 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:01 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:01 INFO DAGScheduler: Submitting ResultStage 94 (MapPartitionsRDD[180] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:49:01 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:01 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:01 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:01 INFO DAGScheduler: Submitting 10 missing tasks from ResultStage 94 (MapPartitionsRDD[180] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Adding task set 94.0 with 10 tasks resource profile 0 23/06/28 07:49:01 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 7670323585580043917, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 7670323585580043917. Created 7670323585580043917 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:49:01 INFO FairSchedulableBuilder: Added task set TaskSet_94.0 tasks to pool 7670323585580043917 23/06/28 07:49:01 INFO TaskSetManager: Starting task 0.0 in stage 94.0 (TID 1245) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 1.0 in stage 94.0 (TID 1246) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 2.0 in stage 94.0 (TID 1247) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 3.0 in stage 94.0 (TID 1248) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 4.0 in stage 94.0 (TID 1249) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 5.0 in stage 94.0 (TID 1250) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 6.0 in stage 94.0 (TID 1251) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 7.0 in stage 94.0 (TID 1252) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 8.0 in stage 94.0 (TID 1253) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 9.0 in stage 94.0 (TID 1254) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_51 stored as values in memory (estimated size 209.9 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_51_piece0 stored as bytes in memory (estimated size 45.1 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on 10.60.24.15:40507 (size: 45.1 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 51 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on 10.60.24.10:41041 (size: 45.1 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on 10.60.24.13:44035 (size: 45.1 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 5.0 in stage 94.0 (TID 1250) in 125 ms on 10.60.24.10 (executor 0) (1/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 9.0 in stage 94.0 (TID 1254) in 125 ms on 10.60.24.10 (executor 0) (2/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 3.0 in stage 94.0 (TID 1248) in 125 ms on 10.60.24.10 (executor 0) (3/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 7.0 in stage 94.0 (TID 1252) in 125 ms on 10.60.24.10 (executor 0) (4/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 2.0 in stage 94.0 (TID 1247) in 128 ms on 10.60.24.13 (executor 1) (5/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 8.0 in stage 94.0 (TID 1253) in 129 ms on 10.60.24.13 (executor 1) (6/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 0.0 in stage 94.0 (TID 1245) in 129 ms on 10.60.24.13 (executor 1) (7/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 4.0 in stage 94.0 (TID 1249) in 129 ms on 10.60.24.13 (executor 1) (8/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 6.0 in stage 94.0 (TID 1251) in 175 ms on 10.60.24.13 (executor 1) (9/10) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 1.0 in stage 94.0 (TID 1246) in 198 ms on 10.60.24.10 (executor 0) (10/10) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Removed TaskSet 94.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:01 INFO DAGScheduler: ResultStage 94 (executeCollect at DatasetRefCache.scala:71) finished in 0.226 s 23/06/28 07:49:01 INFO DAGScheduler: Job 30 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:01 INFO TaskSchedulerImpl: Killing all running tasks in stage 94: Stage finished 23/06/28 07:49:01 INFO DAGScheduler: Job 30 finished: executeCollect at DatasetRefCache.scala:71, took 0.227293 s 23/06/28 07:49:01 INFO FileSourceStrategy: Pushed Filters: IsNotNull(add.dataChange) 23/06/28 07:49:01 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(add#2005.dataChange) 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 27.330251 ms 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_52 stored as values in memory (estimated size 597.3 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in memory (estimated size 16.3 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on 10.60.24.15:40507 (size: 16.3 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 52 from executeCollect at OptimizeLocalUnion.scala:71 23/06/28 07:49:01 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:49:01 INFO SparkContext: Starting job: executeCollect at OptimizeLocalUnion.scala:71 23/06/28 07:49:01 INFO DAGScheduler: Got job 31 (executeCollect at OptimizeLocalUnion.scala:71) with 1 output partitions 23/06/28 07:49:01 INFO DAGScheduler: Final stage: ResultStage 95 (executeCollect at OptimizeLocalUnion.scala:71) 23/06/28 07:49:01 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:01 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:01 INFO DAGScheduler: Submitting ResultStage 95 (UnionRDD[184] at executeCollect at OptimizeLocalUnion.scala:71), which has no missing parents 23/06/28 07:49:01 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:01 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:01 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:01 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 95 (UnionRDD[184] at executeCollect at OptimizeLocalUnion.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Adding task set 95.0 with 1 tasks resource profile 0 23/06/28 07:49:01 INFO FairSchedulableBuilder: Added task set TaskSet_95.0 tasks to pool 7670323585580043917 23/06/28 07:49:01 INFO TaskSetManager: Starting task 0.0 in stage 95.0 (TID 1255) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_53 stored as values in memory (estimated size 254.0 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in memory (estimated size 59.4 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on 10.60.24.15:40507 (size: 59.4 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 53 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on 10.60.24.10:41041 (size: 59.4 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on 10.60.24.10:41041 (size: 16.3 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 0.0 in stage 95.0 (TID 1255) in 153 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Removed TaskSet 95.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:01 INFO DAGScheduler: ResultStage 95 (executeCollect at OptimizeLocalUnion.scala:71) finished in 0.169 s 23/06/28 07:49:01 INFO DAGScheduler: Job 31 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:01 INFO TaskSchedulerImpl: Killing all running tasks in stage 95: Stage finished 23/06/28 07:49:01 INFO DAGScheduler: Job 31 finished: executeCollect at OptimizeLocalUnion.scala:71, took 0.170095 s 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 6.277153 ms 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 7.423753 ms 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 5.302118 ms 23/06/28 07:49:01 INFO DAGScheduler: Registering RDD 187 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 21 23/06/28 07:49:01 INFO DAGScheduler: Got map stage job 32 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 3 output partitions 23/06/28 07:49:01 INFO DAGScheduler: Final stage: ShuffleMapStage 96 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:49:01 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:01 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:01 INFO DAGScheduler: Submitting ShuffleMapStage 96 (MapPartitionsRDD[187] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:49:01 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:01 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:01 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:01 INFO DAGScheduler: Submitting 3 missing tasks from ShuffleMapStage 96 (MapPartitionsRDD[187] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2)) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Adding task set 96.0 with 3 tasks resource profile 0 23/06/28 07:49:01 INFO FairSchedulableBuilder: Added task set TaskSet_96.0 tasks to pool 7670323585580043917 23/06/28 07:49:01 INFO DAGScheduler: Registering RDD 190 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 22 23/06/28 07:49:01 INFO DAGScheduler: Got map stage job 33 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 2 output partitions 23/06/28 07:49:01 INFO DAGScheduler: Final stage: ShuffleMapStage 97 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:49:01 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:01 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:01 INFO TaskSetManager: Starting task 0.0 in stage 96.0 (TID 1256) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 1.0 in stage 96.0 (TID 1257) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 2.0 in stage 96.0 (TID 1258) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO DAGScheduler: Submitting ShuffleMapStage 97 (MapPartitionsRDD[190] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:49:01 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_54 stored as values in memory (estimated size 130.2 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:01 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:01 INFO DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 97 (MapPartitionsRDD[190] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1)) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Adding task set 97.0 with 2 tasks resource profile 0 23/06/28 07:49:01 INFO FairSchedulableBuilder: Added task set TaskSet_97.0 tasks to pool 7670323585580043917 23/06/28 07:49:01 INFO TaskSetManager: Starting task 0.0 in stage 97.0 (TID 1259) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO TaskSetManager: Starting task 1.0 in stage 97.0 (TID 1260) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_55 stored as values in memory (estimated size 130.1 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in memory (estimated size 21.7 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on 10.60.24.15:40507 (size: 21.7 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in memory (estimated size 21.7 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 54 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on 10.60.24.15:40507 (size: 21.7 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 55 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on 10.60.24.10:41041 (size: 21.7 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on 10.60.24.10:41041 (size: 21.7 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on 10.60.24.13:44035 (size: 21.7 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on 10.60.24.13:44035 (size: 21.7 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 2.0 in stage 96.0 (TID 1258) in 21 ms on 10.60.24.10 (executor 0) (1/3) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 0.0 in stage 96.0 (TID 1256) in 21 ms on 10.60.24.10 (executor 0) (2/3) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 1.0 in stage 96.0 (TID 1257) in 22 ms on 10.60.24.13 (executor 1) (3/3) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Removed TaskSet 96.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:01 INFO DAGScheduler: ShuffleMapStage 96 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.026 s 23/06/28 07:49:01 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:01 INFO DAGScheduler: running: Set(ShuffleMapStage 97) 23/06/28 07:49:01 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:01 INFO DAGScheduler: failed: Set() 23/06/28 07:49:01 INFO TaskSetManager: Finished task 0.0 in stage 97.0 (TID 1259) in 24 ms on 10.60.24.10 (executor 0) (1/2) 23/06/28 07:49:01 INFO TaskSetManager: Finished task 1.0 in stage 97.0 (TID 1260) in 26 ms on 10.60.24.13 (executor 1) (2/2) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Removed TaskSet 97.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:01 INFO DAGScheduler: ShuffleMapStage 97 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.027 s 23/06/28 07:49:01 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:01 INFO DAGScheduler: running: Set() 23/06/28 07:49:01 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:01 INFO DAGScheduler: failed: Set() 23/06/28 07:49:01 INFO ShufflePartitionsUtil: For shuffle(21), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:49:01 INFO CodeGenerator: Code generated in 35.969559 ms 23/06/28 07:49:01 INFO DAGScheduler: Registering RDD 198 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 23 23/06/28 07:49:01 INFO DAGScheduler: Got map stage job 34 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:49:01 INFO DAGScheduler: Final stage: ShuffleMapStage 100 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:49:01 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 99, ShuffleMapStage 98) 23/06/28 07:49:01 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:01 INFO DAGScheduler: Submitting ShuffleMapStage 100 (MapPartitionsRDD[198] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:49:01 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:01 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:01 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:01 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 100 (MapPartitionsRDD[198] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:01 INFO TaskSchedulerImpl: Adding task set 100.0 with 1 tasks resource profile 0 23/06/28 07:49:01 INFO FairSchedulableBuilder: Added task set TaskSet_100.0 tasks to pool 7670323585580043917 23/06/28 07:49:01 INFO TaskSetManager: Starting task 0.0 in stage 100.0 (TID 1261) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_56 stored as values in memory (estimated size 209.0 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in memory (estimated size 49.9 KiB, free 750.6 GiB) 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on 10.60.24.15:40507 (size: 49.9 KiB, free: 750.7 GiB) 23/06/28 07:49:01 INFO SparkContext: Created broadcast 56 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:01 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on 10.60.24.13:44035 (size: 49.9 KiB, free: 487.7 GiB) 23/06/28 07:49:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 21 to 10.60.24.13:58548 23/06/28 07:49:01 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 22 to 10.60.24.13:58548 23/06/28 07:49:01 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:49:02 INFO TaskSetManager: Finished task 0.0 in stage 100.0 (TID 1261) in 232 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:49:02 INFO TaskSchedulerImpl: Removed TaskSet 100.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:02 INFO DAGScheduler: ShuffleMapStage 100 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.277 s 23/06/28 07:49:02 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:02 INFO DAGScheduler: running: Set() 23/06/28 07:49:02 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:02 INFO DAGScheduler: failed: Set() 23/06/28 07:49:02 INFO ShufflePartitionsUtil: For shuffle(23), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:49:02 INFO HashAggregateExec: spark.sql.codegen.aggregate.map.twolevel.enabled is set to true, but current version of codegened fast hashmap does not support this aggregate. 23/06/28 07:49:02 INFO CodeGenerator: Code generated in 10.260065 ms 23/06/28 07:49:02 INFO SparkContext: Starting job: executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433 23/06/28 07:49:02 INFO DAGScheduler: Got job 35 (executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433) with 1 output partitions 23/06/28 07:49:02 INFO DAGScheduler: Final stage: ResultStage 104 (executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433) 23/06/28 07:49:02 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 103) 23/06/28 07:49:02 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:02 INFO DAGScheduler: Submitting ResultStage 104 (MapPartitionsRDD[200] at executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433), which has no missing parents 23/06/28 07:49:02 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:02 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:02 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:02 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 104 (MapPartitionsRDD[200] at executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:02 INFO TaskSchedulerImpl: Adding task set 104.0 with 1 tasks resource profile 0 23/06/28 07:49:02 INFO FairSchedulableBuilder: Added task set TaskSet_104.0 tasks to pool 7670323585580043917 23/06/28 07:49:02 INFO TaskSetManager: Starting task 0.0 in stage 104.0 (TID 1262) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_57 stored as values in memory (estimated size 176.3 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_57_piece0 stored as bytes in memory (estimated size 41.0 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on 10.60.24.15:40507 (size: 41.0 KiB, free: 750.7 GiB) 23/06/28 07:49:02 INFO SparkContext: Created broadcast 57 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on 10.60.24.13:44035 (size: 41.0 KiB, free: 487.7 GiB) 23/06/28 07:49:02 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 23 to 10.60.24.13:58548 23/06/28 07:49:02 INFO TaskSetManager: Finished task 0.0 in stage 104.0 (TID 1262) in 56 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:49:02 INFO TaskSchedulerImpl: Removed TaskSet 104.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:02 INFO DAGScheduler: ResultStage 104 (executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433) finished in 0.072 s 23/06/28 07:49:02 INFO DAGScheduler: Job 35 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:02 INFO TaskSchedulerImpl: Killing all running tasks in stage 104: Stage finished 23/06/28 07:49:02 INFO DAGScheduler: Job 35 finished: executeCollect at OptimizeMetadataOnlyDeltaQuery.scala:433, took 0.073486 s 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO CodeGenerator: Code generated in 5.40635 ms 23/06/28 07:49:02 INFO AsyncEventQueue: Process of event SparkListenerJobStart(30,1687938541027,WrappedArray(org.apache.spark.scheduler.StageInfo@4ebe7201),{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem, spark.databricks.session.share=false, spark.hadoop.hive.server2.session.check.interval=60000, spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore, spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.ManagedCatalogClientImpl, spark.driver.port=37693, spark.executor.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djava.io.tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails -verbose:gc -Xss4m -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni -Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl -Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl -Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory -Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -Dscala.reflect.runtime.disable.typetag.cache=true -Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -Ddatabricks.serviceName=spark-executor-1, spark.databricks.clusterUsageTags.sparkMasterUrlType=None, spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false, spark.hadoop.fs.cpfs-s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.databricks.mlflow.autologging.enabled=true, spark.databricks.clusterUsageTags.clusterCreator=Webapp, spark.databricks.clusterUsageTags.enableSqlAclsOnly=false, spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=1, spark.sql.sources.default=delta, spark.openlineage.host=https://functionapptafetst266g.azurewebsites.net, spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStore, spark.driver.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED, spark.databricks.delta.preview.enabled=true, spark.task.reaper.killTimeout=60s, spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.databricks.python.defaultPythonRepl=ipykernel, __fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true, spark.hadoop.fs.s3a.max.total.tasks=1000, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=azure_disk_volume_type: PREMIUM_LRS , databricks.operationType=delta.ddl.showPartitions, spark.home=/databricks/spark, spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.metrics.filesystem_io_metrics=true, spark.hadoop.fs.stage.impl.disable.cache=true, spark.databricks.clusterUsageTags.clusterAvailability=ON_DEMAND_AZURE, spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.DirectParquetOutputCommitter, spark.databricks.io.directoryCommit.enableLogicalDelete=false, spark.app.startTime=1687938293947, spark.executor.tempDirectory=/local_disk0/tmp, spark.hadoop.fs.s3a.connection.maximum=200, spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider, spark.databricks.clusterUsageTags.driverInstanceId=90292f3c1b7d4c3eafe0f39f66343605, spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceautocompute.MACClientImpl, spark.databricks.clusterUsageTags.userProvidedSparkVersion=12.1.x-scala2.12, spark.hadoop.fs.gs.impl.disable.cache=true, spark.databricks.sparkContextId=1758048110757902745, spark.databricks.workerNodeTypeId=Standard_E96ds_v5, spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false, spark.sql.legacy.timeParserPolicy=LEGACY, spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8119356306712193, spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2, spark.databricks.clusterUsageTags.clusterId=0622-000257-omphh17r, spark.databricks.privateLinkEnabled=false, spark.databricks.service.client.type=python, spark.databricks.clusterUsageTags.runtimeEngine=STANDARD, spark.job.description=DeltaSaveDataframeDirect(df_updated, source_sys..., spark.databricks.clusterUsageTags.clusterWorkers=2, spark.app.id=app-20230628074455-0000, spark.r.sql.derby.temp.dir=/tmp/Rtmp5Iz38H, spark.databricks.clusterUsageTags.enableCredentialPassthrough=false, spark.databricks.clusterUsageTags.clusterLastActivityTime=1687937434448, spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databricks.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory, spark.extraListeners=io.openlineage.spark.agent.OpenLineageSparkListener, spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT, spark.databricks.delta.lastCommitVersionInSession=86, spark.hadoop.parquet.page.verify-checksum.enabled=true, spark.databricks.job.runId=5532172, spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.60.24.15, spark.driver.tempDirectory=/local_disk0/tmp, spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-1758048110757902745-2be112e6-25cd-4f64-af80-bd74d906e728, spark.hadoop.fs.wasbs.impl.disable.cache=true, spark.shuffle.service.enabled=true, spark.speculation=false, spark.openlineage.url.param.code=QizqSm8oDrgx83wRKp0j_SlCP8wLcubLcsDaw3Q77WlwAzFuADTImw==, spark.eventLog.enabled=false, spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.databricks.clusterUsageTags.attribute_tag_budget=, spark.databricks.clusterUsageTags.userId=3661563703767194, spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.data.client.MapReduceDirectOutputCommitter, spark.databricks.clusterUsageTags.clusterTargetWorkers=2, spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000, spark.hadoop.fs.adl.impl=com.databricks.adl.AdlFileSystem, spark.sql.hive.convertCTAS=true, spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.driver.credentials.OAuthTokenRefresherClient, spark.databricks.secret.envVar.keys.toRedact=, spark.databricks.clusterUsageTags.containerType=LXC, spark.ui.prometheus.enabled=true, spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true, spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.PersonalStagingFileSystem, spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true, spark.hadoop.fs.s3n.impl.disable.cache=true, spark.hadoop.fs.adl.impl.disable.cache=true, spark.databricks.cloudfetch.hasRegionSupport=true, spark.hadoop.fs.s3a.fast.upload.active.blocks=32, spark.hadoop.fs.s3.impl.disable.cache=true, spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperBridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false, spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=, spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=vnet-injection, spark.databricks.replId=ReplId-6a727-6a976-cc8e8-d, spark.databricks.clusterUsageTags.clusterSpotBidMaxPrice=-1.0, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=0, spark.r.backendConnectionTimeout=604800, spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5, spark.databricks.clusterUsageTags.driverNodeType=Standard_E96ds_v5, spark.sql.catalogImplementation=hive, spark.databricks.clusterUsageTags.clusterGeneration=25, spark.databricks.clusterUsageTags.clusterMaxWorkers=8, spark.databricks.wsfs.workspacePrivatePreview=true, spark.databricks.clusterUsageTags.isSingleUserCluster=false, user=rahul.agrawal1@tafenswtest.edu.au, spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.daemon.data.client.adl.AdlCredentialContextTokenProvider, spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebappScimClient, spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.databricks.unity.TokenServiceApiTokenProvider, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false, spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true, spark.hadoop.parquet.block.size.row.check.min=10, spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils, spark.databricks.enablePublicDbfsFuse=false, spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem, spark.hadoop.fs.wasb.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem, spark.databricks.clusterUsageTags.cloudProvider=Azure, spark.hadoop.hive.server2.thrift.http.port=10000, spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false, spark.shuffle.manager=SORT, spark.hadoop.hive.server2.keystore.password=[REDACTED], spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider, spark.databricks.clusterUsageTags.enableElasticDisk=true, spark.hadoop.parquet.abfs.readahead.optimization.enabled=true, spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30, orgId=8119356306712193, spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DBCEventLoggingListener, spark.databricks.clusterUsageTags.clusterStateMessage=Starting Spark, spark.databricks.clusterUsageTags.sparkVersion=12.1.x-scala2.12, spark.databricks.clusterUsageTags.hailEnabled=false, spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.DatabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*, spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=1, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=0, spark.hadoop.fs.wasbs.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem, spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSystem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false, spark.speculation.multiplier=3, spark.databricks.clusterUsageTags.clusterPythonVersion=3, spark.databricks.clusterUsageTags.ngrokNpipEnabled=false, spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false, spark.databricks.job.type=notebook, spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1, spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(redacted), spark.hadoop.hive.server2.idle.operation.timeout=7200000, spark.hadoop.hive.server2.transport.mode=http, spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30, spark.sql.legacy.parquet.int96RebaseModeInRead=LEGACY, spark.databricks.notebook.path=/build/raw/load-raw-deltatable, spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLogStore, spark.hadoop.fs.s3a.fast.upload.default=true, spark.master=spark://10.60.24.15:7077, spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false, spark.databricks.execution.rootSqlExecutionId=85, spark.files.fetchFailure.unRegisterOutputOnHost=true, spark.hadoop.fs.s3a.retry.throttle.interval=500ms, spark.sql.allowMultipleContexts=false, spark.databricks.clusterUsageTags.privateLinkEnabled=false, spark.hadoop.fs.s3a.fast.upload=true, spark.databricks.secret.sparkConf.keys.toRedact=, spark.executor.memory=65536m, spark.hadoop.fs.azure.user.agent.prefix=, spark.executor.id=driver, spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms, spark.rpc.message.maxSize=256, spark.databricks.token=[REDACTED], spark.databricks.clusterUsageTags.clusterLogDestination=, spark.scheduler.listenerbus.eventqueue.capacity=20000, spark.hadoop.databricks.loki.fileSystemCache.enabled=true, spark.databricks.env=databricks-python-repl-ipykernel, spark.hadoop.parquet.page.size.check.estimate=false, spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true, spark.databricks.wsfsPublicPreview=true, spark.sql.execution.parent=85, spark.rdd.compress=true, spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.client.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000, spark.databricks.driverNodeTypeId=Standard_E96ds_v5, spark.hadoop.fs.azure.skip.metrics=true, spark.hadoop.hive.hmshandler.retry.attempts=10, spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216, spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-Token:spark.databricks.token,X-Databricks-Non-UC-User-Token:spark.databricks.non.uc.token,X-Databricks-Api-Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-Token:spark.databricks.synapse.token,X-Databricks-AWS-Credentials:spark.databricks.aws.creds,X-Databricks-User-Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-Databricks-Oauth-Identity-Custom-Claim:spark.databricks.oauthCustomIdentityClaims, spark.databricks.automl.serviceEnabled=true, spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=true, spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider, spark.akka.frameSize=256, spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daemon.driver.aws.AwsCredentialContextTokenProvider, spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-s3n.impl.disable.cache=true, spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=, spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS, spark.serializer.objectStreamReset=100, spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false, spark.sparkr.use.daemon=false, spark.databricks.clusterUsageTags.clusterFirstOnDemand=1, spark.files.useFetchCache=false, spark.sql.crossJoin.enabled=true, spark.sql.hive.metastore.version=0.13.0, spark.databricks.clusterUsageTags.driverPublicDns=20.5.66.227, spark.databricks.clusterUsageTags.clusterNumCustomTags=0, spark.databricks.clusterUsageTags.clusterName=Databricks-Purview, spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.databricks,com.codahale,com.fasterxml.jackson,shaded.databricks, spark.databricks.clusterUsageTags.clusterNumSshKeys=0, spark.hadoop.fs.s3a.attempts.maximum=10, spark.databricks.clusterUsageTags.orgId=8119356306712193, spark.memory.offHeap.enabled=true, spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy, spark.hadoop.fs.s3a.block.size=67108864, spark.hadoop.hive.server2.enable.doAs=false, spark.openlineage.transport.type=console, spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false, spark.databricks.clusterUsageTags.clusterResourceClass=default, spark.hadoop.hive.server2.idle.session.timeout=900000, spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfss.impl.disable.cache=true, spark.databricks.clusterUsageTags.autoTerminationMinutes=120, spark.databricks.clusterUsageTags.attribute_tag_dust_suite=, spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.backend.daemon.driver.credentials.GlueClientExecutorServiceFactory, spark.databricks.clusterUsageTags.enableDfAcls=false, spark.hadoop.parquet.block.size.row.check.max=10, spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.60.28.15, spark.openlineage.version=v1, spark.databricks.clusterSource=UI, spark.openlineage.facets.disabled=[spark_unknown;], spark.hadoop.fs.cpfs-abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true, spark.databricks.clusterUsageTags.region=australiaeast, spark.ui.port=40001, spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=adb-8119356306712193.13.azuredatabricks.net, spark.hadoop.fs.abfss.impl.disable.cache=true, spark.hadoop.parquet.filter.columnindex.enabled=false, spark.sql.execution.id=86, spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backend.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider, spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-abfss.impl.disable.cache=true, spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem, spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient, spark.databricks.clusterUsageTags.dataPlaneRegion=australiaeast, spark.databricks.clusterUsageTags.clusterOwnerUserId=3661563703767194, spark.streaming.driver.writeAheadLog.allowBatching=true, spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false, spark.hadoop.fs.azure.authorization.caching.enable=false, spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider, spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136, spark.repl.class.uri=spark://10.60.24.15:37693/classes, spark.sql.legacy.parquet.int96RebaseModeInWrite=LEGACY, spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datasources.SQLHadoopMapReduceCommitProtocol, spark.databricks.optimizer.optimizeLimit.maxLimit=1001, spark.driver.host=10.60.24.15, spark.driver.allowMultipleContexts=false, spark.databricks.driverNfs.enabled=true, spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.hadoop.fs.s3a.multipart.size=10485760, spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.task.reaper.enabled=true, spark.databricks.clusterUsageTags.clusterNodeType=Standard_E96ds_v5, spark.databricks.cloudProvider=Azure, spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8119356306712193, spark.shuffle.memoryFraction=0.2, spark.databricks.clusterUsageTags.clusterScalingType=autoscaling, spark.hadoop.fs.fcfs-s3.impl.disable.cache=true, spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1, spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedactorProxyImpl, spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false, spark.app.name=Databricks Shell, spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU, spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheInvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties, spark.databricks.workload.name=JobRun, spark.sql.streaming.stopTimeout=15s, spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider, spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentialsProvider, spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false, spark.files.overwrite=true, spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore, spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScopeFileSystem, spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=false, spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.backend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory, spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true, spark.hadoop.fs.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3, spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3, spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false, spark.shuffle.reduceLocality.enabled=false, spark.databricks.repl.enableClassFileCleanup=true, spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true, spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false, spark.hadoop.databricks.fs.perfMetrics.enable=true, spark.databricks.eventLog.enabled=true, spark.databricks.clusterUsageTags.managedResourceGroup=databricks-rg-ZEDBWBI02T-vrlfjwyjm5sky, spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000, spark.databricks.api.url=https://australiaeast.azuredatabricks.net, spark.databricks.inherited.spark.databricks.api.url=https://australiaeast.azuredatabricks.net, spark.hadoop.fs.fcfs-wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.databricks.clusterUsageTags.clusterPinned=false, spark.speculation.quantile=0.9, spark.databricks.job.id=922321804683480, spark.hadoop.fs.cpfs-s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.hadoop.fs.cpfs-adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, userId=8162102493690770, spark.hadoop.fs.wasb.impl.disable.cache=true, spark.sql.legacy.createHiveTableByDefault=false, spark.openlineage.namespace=adb-8119356306712193.13.#0622-000257-omphh17r, spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048, spark.databricks.clusterUsageTags.isIMv2Enabled=false, spark.hadoop.fs.fcfs-wasb.impl.disable.cache=true, spark.hadoop.databricks.s3.verifyBucketExists.enabled=false, spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs, spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.DirectoryAtomicCommitProtocol, spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true, spark.scheduler.pool=7670323585580043917, spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true, spark.memory.offHeap.size=487180992512, spark.databricks.clusterUsageTags.azureSubscriptionId=f242a3b6-370c-4b2e-971d-a0399c53d6f0, spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false, spark.databricks.clusterUsageTags.clusterMinWorkers=2, spark.databricks.clusterUsageTags.clusterState=Restarting, spark.sql.parquet.cacheMetadata=true, spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER, spark.databricks.clusterUsageTags.effectiveSparkVersion=12.1.x-scala2.12, spark.databricks.clusterUsageTags.driverContainerId=54143fa41c2a41a68bf7dc5182636b69, spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosDbfsReadinessCheckClient, spark.databricks.clusterUsageTags.clusterUnityCatalogMode=CUSTOM, spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"Vendor","value":"Databricks"},{"key":"Creator","value":"chanh.tran@tafenswtest.edu.au"},{"key":"ClusterName","value":"Databricks-Purview"},{"key":"ClusterId","value":"0622-000257-omphh17r"},{"key":"CostCentre","value":"CTP/180529-01"},{"key":"Environment","value":"Test"},{"key":"Owner","value":"Enterprise Data Services"},{"key":"ServiceName","value":"TRIP"},{"key":"DatabricksEnvironment","value":"workerenv-8119356306712193"}], spark.hadoop.parquet.page.write-checksum.enabled=true, spark.cleaner.referenceTracking.blocking=false, spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-s3a.impl.disable.cache=true, spark.databricks.clusterUsageTags.attribute_tag_service=, spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.DataDaemonCloudPresignedUrlRequester, spark.databricks.clusterUsageTags.clusterOwnerOrgId=8119356306712193, spark.jobGroup.id=7670323585580043917_8469927463799253158_job-922321804683480-run-5532172-action-5363477929991621, spark.databricks.workload.id=8119356306712193-922321804683480-5532172, spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServerDBUtils, spark.hadoop.fs.s3a.impl.disable.cache=true, spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false, spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSecretKeyRedactorProxy}) by listener OpenLineageSparkListener took 1.140213765s. 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:49:02 INFO CodeGenerator: Code generated in 7.275276 ms 23/06/28 07:49:02 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:02 INFO ClusterLoadMonitor: Removed query with execution ID:85. Current active queries:0 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:02.193Z","run":{"runId":"1550b6da-2940-48d7-a295-e71bc717893b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.ShowPartitionsDeltaCommand","num-children":0,"deltaTableIdentifier":null,"snapshot":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_show_partitions_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:02 INFO ClusterLoadMonitor: Added query with execution ID:87. Current active queries:1 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:49:02 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:49:02 INFO ParquetUtils: Using user defined output committer for Parquet: org.apache.spark.sql.parquet.DirectParquetOutputCommitter 23/06/28 07:49:02 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:49:02.26Z","run":{"runId":"b8f3c29a-a9e5-4222-b35e-aba23455864b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.CaseWhen","num-children":3,"branches":[{"product-class":"scala.Tuple2","_1":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"_2":[{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]}],"elseValue":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.CaseWhen","num-children":3,"branches":[{"product-class":"scala.Tuple2","_1":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"_2":[{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]}],"elseValue":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.DayOfMonth","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Month","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Year","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.FromUTCTimestamp","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"timestamp","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 07:48:34.707346","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"Australia/Sydney","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":88,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":89,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":90,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":91,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":92,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":93,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":88,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":89,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":90,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":91,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":92,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":93,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":81,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":81,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":33,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":33,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_lmsuu_mdl_groups_members_delta","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"long"},{"name":"groupid","type":"long"},{"name":"userid","type":"long"},{"name":"timeadded","type":"long"},{"name":"component","type":"string"},{"name":"itemid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"year","type":"string"},{"name":"month","type":"string"},{"name":"day","type":"string"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"id"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"groupid"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"userid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"timeadded"}]},"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"component"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"itemid"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:02 INFO CodeGenerator: Code generated in 7.376074 ms 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_58 stored as values in memory (estimated size 519.3 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in memory (estimated size 14.4 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on 10.60.24.15:40507 (size: 14.4 KiB, free: 750.7 GiB) 23/06/28 07:49:02 INFO SparkContext: Created broadcast 58 from write at WriteIntoDeltaCommand.scala:70 23/06/28 07:49:02 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 4194304 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:49:02.483Z","run":{"runId":"f291971f-4742-4b57-89c6-43d0652fbbfc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, groupid, userid, timeadded, component, itemid, _DLRawZoneTimeStamp, year, month, day]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"}],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:48:34.707346","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"06","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"28","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:02 INFO SparkContext: Starting job: write at WriteIntoDeltaCommand.scala:70 23/06/28 07:49:02 INFO DAGScheduler: Got job 36 (write at WriteIntoDeltaCommand.scala:70) with 1 output partitions 23/06/28 07:49:02 INFO DAGScheduler: Final stage: ResultStage 105 (write at WriteIntoDeltaCommand.scala:70) 23/06/28 07:49:02 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:02 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:02 INFO DAGScheduler: Submitting ResultStage 105 (MapPartitionsRDD[204] at write at WriteIntoDeltaCommand.scala:70), which has no missing parents 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:02 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:02 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:02 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:02 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 105 (MapPartitionsRDD[204] at write at WriteIntoDeltaCommand.scala:70) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:02 INFO TaskSchedulerImpl: Adding task set 105.0 with 1 tasks resource profile 0 23/06/28 07:49:02 INFO FairSchedulableBuilder: Added task set TaskSet_105.0 tasks to pool 7670323585580043917 23/06/28 07:49:02 INFO TaskSetManager: Starting task 0.0 in stage 105.0 (TID 1263) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_59 stored as values in memory (estimated size 292.8 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in memory (estimated size 92.5 KiB, free 750.6 GiB) 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on 10.60.24.15:40507 (size: 92.5 KiB, free: 750.7 GiB) 23/06/28 07:49:02 INFO SparkContext: Created broadcast 59 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:02 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:49:02 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:49:02 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:49:02 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on 10.60.24.10:41041 (size: 92.5 KiB, free: 487.7 GiB) 23/06/28 07:49:02 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on 10.60.24.10:41041 (size: 14.4 KiB, free: 487.7 GiB) 23/06/28 07:49:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:49:03 INFO TaskSetManager: Finished task 0.0 in stage 105.0 (TID 1263) in 1205 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:03 INFO TaskSchedulerImpl: Removed TaskSet 105.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:03 INFO DAGScheduler: ResultStage 105 (write at WriteIntoDeltaCommand.scala:70) finished in 1.221 s 23/06/28 07:49:03 INFO DAGScheduler: Job 36 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:03 INFO TaskSchedulerImpl: Killing all running tasks in stage 105: Stage finished 23/06/28 07:49:03 INFO DAGScheduler: Job 36 finished: write at WriteIntoDeltaCommand.scala:70, took 1.223326 s 23/06/28 07:49:03 INFO FileFormatWriter: Start to commit write Job dfb03b1a-76e2-4ee6-af52-2ae4d6b8f173. 23/06/28 07:49:03 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:49:03 INFO FileFormatWriter: Write Job dfb03b1a-76e2-4ee6-af52-2ae4d6b8f173 committed. Elapsed time: 94 ms. 23/06/28 07:49:03 INFO FileFormatWriter: Finished processing stats for write job dfb03b1a-76e2-4ee6-af52-2ae4d6b8f173. 23/06/28 07:49:03 INFO SparkContext: Starting job: collect at TransactionalWriteEdge.scala:576 23/06/28 07:49:03 INFO DAGScheduler: Got job 37 (collect at TransactionalWriteEdge.scala:576) with 1 output partitions 23/06/28 07:49:03 INFO DAGScheduler: Final stage: ResultStage 106 (collect at TransactionalWriteEdge.scala:576) 23/06/28 07:49:03 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:03 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:03 INFO DAGScheduler: Submitting ResultStage 106 (SQLExecutionRDD[206] at toRdd at TransactionalWriteEdge.scala:576), which has no missing parents 23/06/28 07:49:03 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:03 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:03 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:03 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 106 (SQLExecutionRDD[206] at toRdd at TransactionalWriteEdge.scala:576) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:03 INFO TaskSchedulerImpl: Adding task set 106.0 with 1 tasks resource profile 0 23/06/28 07:49:03 INFO FairSchedulableBuilder: Added task set TaskSet_106.0 tasks to pool 7670323585580043917 23/06/28 07:49:03 INFO TaskSetManager: Starting task 0.0 in stage 106.0 (TID 1264) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:03 INFO MemoryStore: Block broadcast_60 stored as values in memory (estimated size 148.8 KiB, free 750.6 GiB) 23/06/28 07:49:03 INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in memory (estimated size 30.0 KiB, free 750.6 GiB) 23/06/28 07:49:03 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on 10.60.24.15:40507 (size: 30.0 KiB, free: 750.7 GiB) 23/06/28 07:49:03 INFO SparkContext: Created broadcast 60 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:03 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on 10.60.24.10:41041 (size: 30.0 KiB, free: 487.7 GiB) 23/06/28 07:49:03 INFO TaskSetManager: Finished task 0.0 in stage 106.0 (TID 1264) in 13 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:03 INFO TaskSchedulerImpl: Removed TaskSet 106.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:03 INFO DAGScheduler: ResultStage 106 (collect at TransactionalWriteEdge.scala:576) finished in 0.015 s 23/06/28 07:49:03 INFO DAGScheduler: Job 37 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:03 INFO TaskSchedulerImpl: Killing all running tasks in stage 106: Stage finished 23/06/28 07:49:03 INFO DAGScheduler: Job 37 finished: collect at TransactionalWriteEdge.scala:576, took 0.016863 s 23/06/28 07:49:03 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:49:03 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:49:04 INFO CodeGenerator: Code generated in 7.76711 ms 23/06/28 07:49:04 INFO SharedState: Maintenance auto compute client is disabled. 23/06/28 07:49:04 INFO OptimisticTransaction: [tableId=32485164,txnId=22cb582e] Attempting to commit version 87 with 3 actions with WriteSerializable isolation level 23/06/28 07:49:04 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:49:04 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp 23/06/28 07:49:04 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp] Closing stream; size: 0 23/06/28 07:49:04 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp] Upload complete; size: 2106 23/06/28 07:49:04 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json] Starting rename. Issuing rename operation. 23/06/28 07:49:04 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json] Rename successful. 23/06/28 07:49:04 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.json.7ae88069-80d3-4e6d-862a-2b9d444b27cb.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json 23/06/28 07:49:04 INFO OptimisticTransaction: [tableId=32485164,txnId=22cb582e] Incremental commit: starting with snapshot version 86 23/06/28 07:49:04 INFO DeltaLog: Loading version 87 starting from checkpoint version 80. 23/06/28 07:49:04 INFO SnapshotEdge: [tableId=32485164-a88a-4b48-ac48-c8898e21a3ee] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=87, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,87,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687938539000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC22229052"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687938544000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC24F5ACF1"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938544000), checksumOpt=Some(VersionChecksum(Some(22cb582e-ee7a-4417-adee-f8c7325a0071),75339,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@1f76b2df,[J@1bb00ea7)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-6ae2224b-e5b9-49aa-887c-1971a09e4d8e.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75339,1687938543000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:48:34.707Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:48:34.707Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687938543000000, MIN_INSERTION_TIME -> 1687938543000000, MAX_INSERTION_TIME -> 1687938543000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_61 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 61 from broadcast at DeltaLog.scala:634 23/06/28 07:49:04 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 27842) 23/06/28 07:49:04 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 7, totalFileSize: 9388) 23/06/28 07:49:04 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:49:04 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_62 stored as values in memory (estimated size 609.9 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 62 from $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173 23/06/28 07:49:04 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:49:04 INFO DAGScheduler: Registering RDD 213 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 24 23/06/28 07:49:04 INFO DAGScheduler: Got map stage job 38 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 14 output partitions 23/06/28 07:49:04 INFO DAGScheduler: Final stage: ShuffleMapStage 107 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:49:04 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:04 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:04 INFO DAGScheduler: Submitting ShuffleMapStage 107 (MapPartitionsRDD[213] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:49:04 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:04 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:04 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:04 INFO DAGScheduler: Submitting 14 missing tasks from ShuffleMapStage 107 (MapPartitionsRDD[213] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Adding task set 107.0 with 14 tasks resource profile 0 23/06/28 07:49:04 INFO FairSchedulableBuilder: Added task set TaskSet_107.0 tasks to pool 7670323585580043917 23/06/28 07:49:04 INFO TaskSetManager: Starting task 0.0 in stage 107.0 (TID 1265) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 1.0 in stage 107.0 (TID 1266) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 2.0 in stage 107.0 (TID 1267) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 3.0 in stage 107.0 (TID 1268) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 4.0 in stage 107.0 (TID 1269) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 5.0 in stage 107.0 (TID 1270) (10.60.24.13, executor 1, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 6.0 in stage 107.0 (TID 1271) (10.60.24.10, executor 0, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 7.0 in stage 107.0 (TID 1272) (10.60.24.13, executor 1, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 8.0 in stage 107.0 (TID 1273) (10.60.24.10, executor 0, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 9.0 in stage 107.0 (TID 1274) (10.60.24.13, executor 1, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 10.0 in stage 107.0 (TID 1275) (10.60.24.10, executor 0, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 11.0 in stage 107.0 (TID 1276) (10.60.24.13, executor 1, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 12.0 in stage 107.0 (TID 1277) (10.60.24.10, executor 0, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO TaskSetManager: Starting task 13.0 in stage 107.0 (TID 1278) (10.60.24.13, executor 1, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_63 stored as values in memory (estimated size 324.4 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in memory (estimated size 79.4 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on 10.60.24.15:40507 (size: 79.4 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 63 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on 10.60.24.13:44035 (size: 79.4 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on 10.60.24.10:41041 (size: 79.4 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 2.0 in stage 107.0 (TID 1267) in 18 ms on 10.60.24.10 (executor 0) (1/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 6.0 in stage 107.0 (TID 1271) in 19 ms on 10.60.24.10 (executor 0) (2/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 8.0 in stage 107.0 (TID 1273) in 19 ms on 10.60.24.10 (executor 0) (3/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 10.0 in stage 107.0 (TID 1275) in 19 ms on 10.60.24.10 (executor 0) (4/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 12.0 in stage 107.0 (TID 1277) in 19 ms on 10.60.24.10 (executor 0) (5/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 4.0 in stage 107.0 (TID 1269) in 19 ms on 10.60.24.10 (executor 0) (6/14) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 5.0 in stage 107.0 (TID 1270) in 19 ms on 10.60.24.13 (executor 1) (7/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 11.0 in stage 107.0 (TID 1276) in 19 ms on 10.60.24.13 (executor 1) (8/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 7.0 in stage 107.0 (TID 1272) in 20 ms on 10.60.24.13 (executor 1) (9/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 3.0 in stage 107.0 (TID 1268) in 20 ms on 10.60.24.13 (executor 1) (10/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 13.0 in stage 107.0 (TID 1278) in 20 ms on 10.60.24.13 (executor 1) (11/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 1.0 in stage 107.0 (TID 1266) in 20 ms on 10.60.24.13 (executor 1) (12/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 9.0 in stage 107.0 (TID 1274) in 20 ms on 10.60.24.13 (executor 1) (13/14) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 0.0 in stage 107.0 (TID 1265) in 45 ms on 10.60.24.10 (executor 0) (14/14) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Removed TaskSet 107.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:04 INFO DAGScheduler: ShuffleMapStage 107 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.054 s 23/06/28 07:49:04 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:04 INFO DAGScheduler: running: Set() 23/06/28 07:49:04 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:04 INFO DAGScheduler: failed: Set() 23/06/28 07:49:04 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":6027,"lifetime":243104},"10.60.24.13":{"diskUsage":0,"lifetime":243374}}, ioCacheNumScanTasks={"numLocalScanTasks":5,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:49:04 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:49:04 INFO DAGScheduler: Got job 39 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:49:04 INFO DAGScheduler: Final stage: ResultStage 109 (collect at Checksum.scala:561) 23/06/28 07:49:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 108) 23/06/28 07:49:04 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:04 INFO DAGScheduler: Submitting ResultStage 109 (MapPartitionsRDD[224] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:49:04 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:04 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:04 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:04 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 109 (MapPartitionsRDD[224] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Adding task set 109.0 with 1 tasks resource profile 0 23/06/28 07:49:04 INFO FairSchedulableBuilder: Added task set TaskSet_109.0 tasks to pool 7670323585580043917 23/06/28 07:49:04 INFO TaskSetManager: Starting task 0.0 in stage 109.0 (TID 1279) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_64 stored as values in memory (estimated size 502.8 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in memory (estimated size 126.8 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on 10.60.24.15:40507 (size: 126.8 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 64 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on 10.60.24.10:41041 (size: 126.8 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 24 to 10.60.24.10:44288 23/06/28 07:49:04 INFO BlockManagerInfo: Added rdd_220_0 in memory on 10.60.24.10:41041 (size: 2.4 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 0.0 in stage 109.0 (TID 1279) in 61 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Removed TaskSet 109.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:04 INFO DAGScheduler: ResultStage 109 (collect at Checksum.scala:561) finished in 0.091 s 23/06/28 07:49:04 INFO DAGScheduler: Job 39 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:04 INFO TaskSchedulerImpl: Killing all running tasks in stage 109: Stage finished 23/06/28 07:49:04 INFO DAGScheduler: Job 39 finished: collect at Checksum.scala:561, took 0.092186 s 23/06/28 07:49:04 INFO DAGScheduler: Registering RDD 227 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 25 23/06/28 07:49:04 INFO DAGScheduler: Got map stage job 40 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:49:04 INFO DAGScheduler: Final stage: ShuffleMapStage 111 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:49:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 110) 23/06/28 07:49:04 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:04 INFO DAGScheduler: Submitting ShuffleMapStage 111 (MapPartitionsRDD[227] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:49:04 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:04 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:04 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:04 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 111 (MapPartitionsRDD[227] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Adding task set 111.0 with 1 tasks resource profile 0 23/06/28 07:49:04 INFO FairSchedulableBuilder: Added task set TaskSet_111.0 tasks to pool 7670323585580043917 23/06/28 07:49:04 INFO TaskSetManager: Starting task 0.0 in stage 111.0 (TID 1280) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_65 stored as values in memory (estimated size 473.1 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in memory (estimated size 120.7 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on 10.60.24.15:40507 (size: 120.7 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 65 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on 10.60.24.10:41041 (size: 120.7 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO TaskSetManager: Finished task 0.0 in stage 111.0 (TID 1280) in 30 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Removed TaskSet 111.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:04 INFO DAGScheduler: ShuffleMapStage 111 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.044 s 23/06/28 07:49:04 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:04 INFO DAGScheduler: running: Set() 23/06/28 07:49:04 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:04 INFO DAGScheduler: failed: Set() 23/06/28 07:49:04 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:49:04 INFO DAGScheduler: Got job 41 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:49:04 INFO DAGScheduler: Final stage: ResultStage 114 (first at Snapshot.scala:271) 23/06/28 07:49:04 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 113) 23/06/28 07:49:04 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:04 INFO DAGScheduler: Submitting ResultStage 114 (MapPartitionsRDD[229] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:49:04 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:04 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:04 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:04 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 114 (MapPartitionsRDD[229] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Adding task set 114.0 with 1 tasks resource profile 0 23/06/28 07:49:04 INFO FairSchedulableBuilder: Added task set TaskSet_114.0 tasks to pool 7670323585580043917 23/06/28 07:49:04 INFO TaskSetManager: Starting task 0.0 in stage 114.0 (TID 1281) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_66 stored as values in memory (estimated size 419.3 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in memory (estimated size 110.2 KiB, free 750.6 GiB) 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on 10.60.24.15:40507 (size: 110.2 KiB, free: 750.7 GiB) 23/06/28 07:49:04 INFO SparkContext: Created broadcast 66 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:04 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on 10.60.24.13:44035 (size: 110.2 KiB, free: 487.7 GiB) 23/06/28 07:49:04 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 25 to 10.60.24.13:58548 23/06/28 07:49:04 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:49:04 INFO TaskSetManager: Finished task 0.0 in stage 114.0 (TID 1281) in 39 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:49:04 INFO TaskSchedulerImpl: Removed TaskSet 114.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:04 INFO DAGScheduler: ResultStage 114 (first at Snapshot.scala:271) finished in 0.061 s 23/06/28 07:49:04 INFO DAGScheduler: Job 41 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:04 INFO TaskSchedulerImpl: Killing all running tasks in stage 114: Stage finished 23/06/28 07:49:04 INFO DAGScheduler: Job 41 finished: first at Snapshot.scala:271, took 0.062314 s 23/06/28 07:49:04 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log, version=87, metadata=Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)), logSegment=LogSegment(dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log,87,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000081.json; isDirectory=false; length=2105; replication=1; blocksize=268435456; modification_time=1687852660000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42E30DCCB'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000082.json; isDirectory=false; length=792; replication=1; blocksize=268435456; modification_time=1687937139000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8DF9C04AA'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000083.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687937144000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E2E1199B'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000084.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687937151000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A8E6E434CE'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000085.json; isDirectory=false; length=787; replication=1; blocksize=268435456; modification_time=1687938535000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC2012D65C"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000086.json; isDirectory=false; length=746; replication=1; blocksize=268435456; modification_time=1687938539000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC22229052"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.json; isDirectory=false; length=2106; replication=1; blocksize=268435456; modification_time=1687938544000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC24F5ACF1"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000080.checkpoint.parquet; isDirectory=false; length=27842; replication=1; blocksize=268435456; modification_time=1687852645000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB76E42590BE0E'}),Some(80),1687938544000), checksumOpt=Some(VersionChecksum(Some(22cb582e-ee7a-4417-adee-f8c7325a0071),75339,1,None,None,1,1,Some(Stream()),Metadata(32485164-a88a-4b48-ac48-c8898e21a3ee,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},List(year, month, day),Map(),Some(1642649430283)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@1f76b2df,[J@1bb00ea7)),None,Some(Stream(AddFile(year=2023/month=06/day=28/part-00000-6ae2224b-e5b9-49aa-887c-1971a09e4d8e.c000.snappy.parquet,Map(year -> 2023, month -> 06, day -> 28),75339,1687938543000,false,{"numRecords":5708,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2023-06-28T17:48:34.707Z"},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2023-06-28T17:48:34.707Z"},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0}},Map(INSERTION_TIME -> 1687938543000000, MIN_INSERTION_TIME -> 1687938543000000, MAX_INSERTION_TIME -> 1687938543000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:49:04 INFO MapPartitionsRDD: Removing RDD 168 from persistence list 23/06/28 07:49:04 INFO BlockManager: Removing RDD 168 23/06/28 07:49:04 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:49:04 INFO OptimisticTransaction: [tableId=32485164,txnId=22cb582e] Committed delta #87 to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log 23/06/28 07:49:04 INFO ChecksumHook: Writing checksum file for table path dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log version 87 23/06/28 07:49:04 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:49:05 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.crc using temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp 23/06/28 07:49:05 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp] Closing stream; size: 0 23/06/28 07:49:05 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp] Upload complete; size: 3471 23/06/28 07:49:05 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.crc] Starting rename. Issuing rename operation. 23/06/28 07:49:05 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp] DST[abfss://raw@zedlsbi01t.dfs.core.windows.net/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.crc] Rename successful. 23/06/28 07:49:05 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000087.crc.512158e5-1930-4dc9-97a6-f3ea4de293d2.tmp to dbfs:/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000087.crc 23/06/28 07:49:05 INFO ClusterLoadMonitor: Removed query with execution ID:87. Current active queries:0 23/06/28 07:49:05 INFO HiveMetaStore: 1: get_database: raw 23/06/28 07:49:05 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:49:05 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:05 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:05 INFO ClusterLoadMonitor: Added query with execution ID:93. Current active queries:1 23/06/28 07:49:05 INFO HiveMetaStore: 1: get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:05 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=raw tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:49:05 INFO ClusterLoadMonitor: Removed query with execution ID:93. Current active queries:0 23/06/28 07:49:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_8469927463799253158_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_9030126128348387620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_9030126128348387620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:05 INFO ClusterLoadMonitor: Added query with execution ID:94. Current active queries:1 23/06/28 07:49:05 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:49:05 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_67 stored as values in memory (estimated size 519.3 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in memory (estimated size 14.4 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on 10.60.24.15:40507 (size: 14.4 KiB, free: 750.7 GiB) 23/06/28 07:49:05 INFO SparkContext: Created broadcast 67 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:49:05 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 4194304 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:49:05 INFO DAGScheduler: Registering RDD 233 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 26 23/06/28 07:49:05 INFO DAGScheduler: Got map stage job 42 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:49:05 INFO DAGScheduler: Final stage: ShuffleMapStage 115 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:49:05 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:49:05 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:05 INFO DAGScheduler: Submitting ShuffleMapStage 115 (MapPartitionsRDD[233] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:49:05 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:05 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:05 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:05 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 115 (MapPartitionsRDD[233] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:05 INFO TaskSchedulerImpl: Adding task set 115.0 with 1 tasks resource profile 0 23/06/28 07:49:05 INFO FairSchedulableBuilder: Added task set TaskSet_115.0 tasks to pool 7670323585580043917 23/06/28 07:49:05 INFO TaskSetManager: Starting task 0.0 in stage 115.0 (TID 1282) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_68 stored as values in memory (estimated size 142.9 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in memory (estimated size 27.0 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on 10.60.24.15:40507 (size: 27.0 KiB, free: 750.7 GiB) 23/06/28 07:49:05 INFO SparkContext: Created broadcast 68 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on 10.60.24.10:41041 (size: 27.0 KiB, free: 487.7 GiB) 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on 10.60.24.10:41041 (size: 14.4 KiB, free: 487.7 GiB) 23/06/28 07:49:05 INFO TaskSetManager: Finished task 0.0 in stage 115.0 (TID 1282) in 82 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:05 INFO TaskSchedulerImpl: Removed TaskSet 115.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:05 INFO DAGScheduler: ShuffleMapStage 115 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.086 s 23/06/28 07:49:05 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:49:05 INFO DAGScheduler: running: Set() 23/06/28 07:49:05 INFO DAGScheduler: waiting: Set() 23/06/28 07:49:05 INFO DAGScheduler: failed: Set() 23/06/28 07:49:05 INFO SparkContext: Starting job: count at NativeMethodAccessorImpl.java:0 23/06/28 07:49:05 INFO DAGScheduler: Got job 43 (count at NativeMethodAccessorImpl.java:0) with 1 output partitions 23/06/28 07:49:05 INFO DAGScheduler: Final stage: ResultStage 117 (count at NativeMethodAccessorImpl.java:0) 23/06/28 07:49:05 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 116) 23/06/28 07:49:05 INFO DAGScheduler: Missing parents: List() 23/06/28 07:49:05 INFO DAGScheduler: Submitting ResultStage 117 (MapPartitionsRDD[235] at count at NativeMethodAccessorImpl.java:0), which has no missing parents 23/06/28 07:49:05 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:49:05 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:49:05 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:49:05 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 117 (MapPartitionsRDD[235] at count at NativeMethodAccessorImpl.java:0) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:49:05 INFO TaskSchedulerImpl: Adding task set 117.0 with 1 tasks resource profile 0 23/06/28 07:49:05 INFO FairSchedulableBuilder: Added task set TaskSet_117.0 tasks to pool 7670323585580043917 23/06/28 07:49:05 INFO TaskSetManager: Starting task 0.0 in stage 117.0 (TID 1283) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_69 stored as values in memory (estimated size 129.8 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in memory (estimated size 21.3 KiB, free 750.6 GiB) 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on 10.60.24.15:40507 (size: 21.3 KiB, free: 750.7 GiB) 23/06/28 07:49:05 INFO SparkContext: Created broadcast 69 from broadcast at TaskSetManager.scala:627 23/06/28 07:49:05 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on 10.60.24.10:41041 (size: 21.3 KiB, free: 487.7 GiB) 23/06/28 07:49:05 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 26 to 10.60.24.10:44288 23/06/28 07:49:05 INFO TaskSetManager: Finished task 0.0 in stage 117.0 (TID 1283) in 15 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:49:05 INFO TaskSchedulerImpl: Removed TaskSet 117.0, whose tasks have all completed, from pool 7670323585580043917 23/06/28 07:49:05 INFO DAGScheduler: ResultStage 117 (count at NativeMethodAccessorImpl.java:0) finished in 0.018 s 23/06/28 07:49:05 INFO DAGScheduler: Job 43 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:49:05 INFO TaskSchedulerImpl: Killing all running tasks in stage 117: Stage finished 23/06/28 07:49:05 INFO DAGScheduler: Job 43 finished: count at NativeMethodAccessorImpl.java:0, took 0.018664 s 23/06/28 07:49:05 INFO ClusterLoadMonitor: Removed query with execution ID:94. Current active queries:0 23/06/28 07:49:05 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_9030126128348387620_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:05 INFO ProgressReporter$: Removed result fetcher for 7285677779516333513_5356253227447830275_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:05 INFO ProgressReporter$: Added result fetcher for 7670323585580043917_5356253227447830275_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:06 INFO ProgressReporter$: Removed result fetcher for 7670323585580043917_5356253227447830275_job-922321804683480-run-5532172-action-5363477929991621 23/06/28 07:49:07 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:49:10 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:49:11 INFO DriverCorral$: Cleaning the wrapper ReplId-6a727-6a976-cc8e8-d (currently in status Idle(ReplId-6a727-6a976-cc8e8-d)) 23/06/28 07:49:11 INFO DriverCorral$: sending shutdown signal for REPL ReplId-6a727-6a976-cc8e8-d 23/06/28 07:49:11 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:49:11 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:49:11 INFO ReplCrashUtils$: python shell exit code: 143 23/06/28 07:49:11 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-6a727-6a976-cc8e8-d 23/06/28 07:49:11 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-6a727-6a976-cc8e8-d 23/06/28 07:49:11 INFO DriverCorral$: ReplId-6a727-6a976-cc8e8-d successfully discarded 23/06/28 07:49:11 INFO DriverCorral$: Cleaning the wrapper ReplId-651be-d56dc-071dc-9 (currently in status Idle(ReplId-651be-d56dc-071dc-9)) 23/06/28 07:49:11 INFO DriverCorral$: sending shutdown signal for REPL ReplId-651be-d56dc-071dc-9 23/06/28 07:49:11 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-651be-d56dc-071dc-9 23/06/28 07:49:11 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-651be-d56dc-071dc-9 23/06/28 07:49:11 INFO DriverCorral$: ReplId-651be-d56dc-071dc-9 successfully discarded 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:49:02.617Z","run":{"runId":"f291971f-4742-4b57-89c6-43d0652fbbfc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, groupid, userid, timeadded, component, itemid, _DLRawZoneTimeStamp, year, month, day]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"}],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:48:34.707346","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"06","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"28","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532172","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/raw/load-raw-deltatable","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"922321804683480","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO AsyncEventQueue: Process of event SparkListenerJobStart(36,1687938542617,WrappedArray(org.apache.spark.scheduler.StageInfo@24895d2b),{spark.hadoop.fs.idbfs.impl=com.databricks.io.idbfs.IdbfsFileSystem, spark.databricks.session.share=false, spark.hadoop.hive.server2.session.check.interval=60000, spark.databricks.tahoe.logStore.class=com.databricks.tahoe.store.DelegatingLogStore, spark.databricks.managedCatalog.clientClassName=com.databricks.managedcatalog.ManagedCatalogClientImpl, spark.driver.port=37693, spark.executor.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djava.io.tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=512m -XX:+UseCodeCacheFlushing -XX:PerMethodRecompilationCutoff=-1 -XX:PerBytecodeRecompilationCutoff=-1 -Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:-UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -XX:+PrintGCDetails -verbose:gc -Xss4m -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni -Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl -Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl -Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory -Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl -Djavax.net.ssl.sessionCacheSize=10000 -Dscala.reflect.runtime.disable.typetag.cache=true -Dcom.google.cloud.spark.bigquery.repackaged.io.netty.tryReflectionSetAccessible=true -Dlog4j2.formatMsgNoLookups=true -Ddatabricks.serviceName=spark-executor-1, spark.databricks.clusterUsageTags.sparkMasterUrlType=None, spark.databricks.clusterUsageTags.sparkEnvVarContainsSingleQuotes=false, spark.hadoop.fs.cpfs-s3n.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.databricks.mlflow.autologging.enabled=true, spark.databricks.clusterUsageTags.clusterCreator=Webapp, spark.databricks.clusterUsageTags.enableSqlAclsOnly=false, spark.databricks.clusterUsageTags.numPerClusterInitScriptsV2=1, spark.sql.sources.default=delta, spark.openlineage.host=https://functionapptafetst266g.azurewebsites.net, spark.databricks.tahoe.logStore.azure.class=com.databricks.tahoe.store.AzureLogStore, spark.driver.extraJavaOptions=-XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED, spark.databricks.delta.preview.enabled=true, spark.task.reaper.killTimeout=60s, spark.hadoop.fs.fcfs-s3.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.databricks.python.defaultPythonRepl=ipykernel, __fetch_continuous_blocks_in_batch_enabled=true, spark.job.interruptOnCancel=true, spark.hadoop.fs.s3a.max.total.tasks=1000, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeType=azure_disk_volume_type: PREMIUM_LRS , databricks.operationType=delta.transaction.writeFiles, spark.home=/databricks/spark, spark.hadoop.fs.s3n.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.metrics.filesystem_io_metrics=true, spark.hadoop.fs.stage.impl.disable.cache=true, spark.databricks.clusterUsageTags.clusterAvailability=ON_DEMAND_AZURE, spark.hadoop.spark.sql.parquet.output.committer.class=org.apache.spark.sql.parquet.DirectParquetOutputCommitter, spark.databricks.io.directoryCommit.enableLogicalDelete=false, spark.app.startTime=1687938293947, spark.executor.tempDirectory=/local_disk0/tmp, spark.hadoop.fs.s3a.connection.maximum=200, spark.databricks.credential.scope.fs.adls.gen2.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeADLSTokenProvider, spark.databricks.clusterUsageTags.driverInstanceId=90292f3c1b7d4c3eafe0f39f66343605, spark.databricks.autotune.maintenance.client.classname=com.databricks.maintenanceautocompute.MACClientImpl, spark.databricks.clusterUsageTags.userProvidedSparkVersion=12.1.x-scala2.12, spark.hadoop.fs.gs.impl.disable.cache=true, spark.databricks.sparkContextId=1758048110757902745, spark.databricks.workerNodeTypeId=Standard_E96ds_v5, spark.hadoop.databricks.s3.create.deleteUnnecessaryFakeDirectories=false, spark.sql.legacy.timeParserPolicy=LEGACY, spark.databricks.clusterUsageTags.workerEnvironmentId=workerenv-8119356306712193, spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2, spark.databricks.clusterUsageTags.clusterId=0622-000257-omphh17r, spark.databricks.privateLinkEnabled=false, spark.databricks.service.client.type=python, spark.databricks.clusterUsageTags.runtimeEngine=STANDARD, spark.job.description=DeltaSaveDataframeDirect(df_updated, source_sys..., spark.databricks.clusterUsageTags.clusterWorkers=2, spark.app.id=app-20230628074455-0000, spark.r.sql.derby.temp.dir=/tmp/Rtmp5Iz38H, spark.databricks.clusterUsageTags.enableCredentialPassthrough=false, spark.databricks.clusterUsageTags.clusterLastActivityTime=1687937434448, spark.databricks.passthrough.glue.credentialsProviderFactoryClassName=com.databricks.backend.daemon.driver.credentials.DatabricksCredentialProviderFactory, spark.extraListeners=io.openlineage.spark.agent.OpenLineageSparkListener, spark.databricks.clusterUsageTags.clusterMetastoreAccessType=RDS_DIRECT, spark.databricks.delta.lastCommitVersionInSession=86, spark.hadoop.parquet.page.verify-checksum.enabled=true, spark.databricks.job.runId=5532172, spark.databricks.clusterUsageTags.driverContainerPrivateIp=10.60.24.15, spark.driver.tempDirectory=/local_disk0/tmp, spark.repl.class.outputDir=/local_disk0/tmp/repl/spark-1758048110757902745-2be112e6-25cd-4f64-af80-bd74d906e728, spark.hadoop.fs.wasbs.impl.disable.cache=true, spark.shuffle.service.enabled=true, spark.speculation=false, spark.openlineage.url.param.code=QizqSm8oDrgx83wRKp0j_SlCP8wLcubLcsDaw3Q77WlwAzFuADTImw==, spark.eventLog.enabled=false, spark.databricks.clusterUsageTags.enableJdbcAutoStart=true, spark.hadoop.fs.fcfs-s3n.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sql.hive.metastore.jars=/databricks/databricks-hive/*, spark.hadoop.fs.cpfs-s3.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.databricks.clusterUsageTags.attribute_tag_budget=, spark.databricks.clusterUsageTags.userId=3661563703767194, spark.hadoop.spark.sql.sources.outputCommitterClass=com.databricks.backend.daemon.data.client.MapReduceDirectOutputCommitter, spark.databricks.clusterUsageTags.clusterTargetWorkers=2, spark.hadoop.spark.hadoop.aws.glue.cache.db.size=1000, spark.hadoop.fs.adl.impl=com.databricks.adl.AdlFileSystem, spark.sql.hive.convertCTAS=true, spark.databricks.passthrough.oauth.refresher.impl=com.databricks.backend.daemon.driver.credentials.OAuthTokenRefresherClient, spark.databricks.secret.envVar.keys.toRedact=, spark.databricks.clusterUsageTags.containerType=LXC, spark.ui.prometheus.enabled=true, spark.worker.cleanup.enabled=false, spark.sql.hive.convertMetastoreParquet=true, spark.hadoop.fs.stage.impl=com.databricks.backend.daemon.driver.managedcatalog.PersonalStagingFileSystem, spark.hadoop.fs.s3.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.clusterUsageTags.enableJobsAutostart=true, spark.hadoop.fs.s3n.impl.disable.cache=true, spark.hadoop.fs.adl.impl.disable.cache=true, spark.databricks.cloudfetch.hasRegionSupport=true, spark.hadoop.fs.s3a.fast.upload.active.blocks=32, spark.hadoop.fs.s3.impl.disable.cache=true, spark.databricks.sql.configMapperClass=com.databricks.dbsql.config.SqlConfigMapperBridge, spark.hadoop.databricks.s3commit.client.sslTrustAll=false, spark.databricks.clusterUsageTags.attribute_tag_dust_maintainer=, spark.databricks.clusterUsageTags.instanceWorkerEnvNetworkType=vnet-injection, spark.databricks.replId=ReplId-6a727-6a976-cc8e8-d, spark.databricks.clusterUsageTags.clusterSpotBidMaxPrice=-1.0, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeCount=0, spark.r.backendConnectionTimeout=604800, spark.driver.maxResultSize=4g, spark.hadoop.parquet.memory.pool.ratio=0.5, spark.databricks.clusterUsageTags.driverNodeType=Standard_E96ds_v5, spark.sql.catalogImplementation=hive, spark.databricks.clusterUsageTags.clusterGeneration=25, spark.databricks.clusterUsageTags.clusterMaxWorkers=8, spark.databricks.wsfs.workspacePrivatePreview=true, spark.databricks.clusterUsageTags.isSingleUserCluster=false, user=rahul.agrawal1@tafenswtest.edu.au, spark.databricks.passthrough.adls.tokenProviderClassName=com.databricks.backend.daemon.data.client.adl.AdlCredentialContextTokenProvider, spark.databricks.acl.scim.client=com.databricks.spark.sql.acl.client.DriverToWebappScimClient, spark.databricks.unityCatalog.credentialManager.apiTokenProviderClassName=com.databricks.unity.TokenServiceApiTokenProvider, spark.databricks.clusterUsageTags.isServicePrincipalCluster=false, spark.sql.warehouse.dir=dbfs:/user/hive/warehouse, spark.logConf=true, spark.hadoop.parquet.block.size.row.check.min=10, spark.databricks.service.dbutils.repl.backend=com.databricks.dbconnect.ReplDBUtils, spark.databricks.enablePublicDbfsFuse=false, spark.hadoop.fs.abfss.impl=com.databricks.common.filesystem.LokiFileSystem, spark.hadoop.fs.wasb.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem, spark.databricks.clusterUsageTags.cloudProvider=Azure, spark.hadoop.hive.server2.thrift.http.port=10000, spark.databricks.clusterUsageTags.clusterLogDeliveryEnabled=false, spark.shuffle.manager=SORT, spark.hadoop.hive.server2.keystore.password=[REDACTED], spark.databricks.acl.provider=com.databricks.sql.acl.ReflectionBackedAclProvider, spark.databricks.clusterUsageTags.enableElasticDisk=true, spark.hadoop.parquet.abfs.readahead.optimization.enabled=true, spark.hadoop.spark.hadoop.aws.glue.cache.table.ttl-mins=30, orgId=8119356306712193, spark.databricks.eventLog.listenerClassName=com.databricks.backend.daemon.driver.DBCEventLoggingListener, spark.databricks.clusterUsageTags.clusterStateMessage=Starting Spark, spark.databricks.clusterUsageTags.sparkVersion=12.1.x-scala2.12, spark.databricks.clusterUsageTags.hailEnabled=false, spark.sql.streaming.checkpointFileManagerClass=com.databricks.spark.sql.streaming.DatabricksCheckpointFileManager, spark.executor.extraClassPath=/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/*, spark.databricks.clusterUsageTags.numPerGlobalInitScriptsV2=1, spark.databricks.clusterUsageTags.userProvidedRemoteVolumeSizeGb=0, spark.hadoop.fs.wasbs.impl=shaded.databricks.org.apache.hadoop.fs.azure.NativeAzureFileSystem, spark.hadoop.fs.file.impl=com.databricks.backend.daemon.driver.WorkspaceLocalFileSystem, spark.hadoop.hive.warehouse.subdir.inherit.perms=false, spark.speculation.multiplier=3, spark.databricks.clusterUsageTags.clusterPythonVersion=3, spark.databricks.clusterUsageTags.ngrokNpipEnabled=false, spark.hadoop.hive.server2.thrift.http.cookie.auth.enabled=false, spark.databricks.job.type=notebook, spark.storage.memoryFraction=0.5, spark.r.numRBackendThreads=1, spark.databricks.inherited.credentials.keys.spark.databricks.api.token=*********(redacted), spark.hadoop.hive.server2.idle.operation.timeout=7200000, spark.hadoop.hive.server2.transport.mode=http, spark.hadoop.spark.hadoop.aws.glue.cache.db.ttl-mins=30, spark.sql.legacy.parquet.int96RebaseModeInRead=LEGACY, spark.databricks.notebook.path=/build/raw/load-raw-deltatable, spark.databricks.tahoe.logStore.aws.class=com.databricks.tahoe.store.MultiClusterLogStore, spark.hadoop.fs.s3a.fast.upload.default=true, spark.master=spark://10.60.24.15:7077, spark.databricks.clusterUsageTags.sparkEnvVarContainsNewline=false, spark.databricks.execution.rootSqlExecutionId=87, spark.files.fetchFailure.unRegisterOutputOnHost=true, spark.hadoop.fs.s3a.retry.throttle.interval=500ms, spark.sql.allowMultipleContexts=false, spark.databricks.clusterUsageTags.privateLinkEnabled=false, spark.hadoop.fs.s3a.fast.upload=true, spark.databricks.secret.sparkConf.keys.toRedact=, spark.executor.memory=65536m, spark.hadoop.fs.azure.user.agent.prefix=, spark.executor.id=driver, spark.worker.aioaLazyConfig.iamReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosIamRoleCheckClient, spark.hadoop.fs.s3a.retry.interval=250ms, spark.rpc.message.maxSize=256, spark.databricks.token=REDACTED_API_TOKEN, spark.databricks.clusterUsageTags.clusterLogDestination=, spark.scheduler.listenerbus.eventqueue.capacity=20000, spark.hadoop.databricks.loki.fileSystemCache.enabled=true, spark.databricks.env=databricks-python-repl-ipykernel, spark.hadoop.parquet.page.size.check.estimate=false, spark.hadoop.hive.server2.keystore.path=/databricks/keys/jetty-ssl-driver-keystore.jks, spark.streaming.driver.writeAheadLog.closeFileAfterWrite=true, spark.rdd.scope.noOverride=true, spark.databricks.wsfsPublicPreview=true, spark.sql.execution.parent=87, spark.rdd.compress=true, spark.hadoop.mapred.output.committer.class=com.databricks.backend.daemon.data.client.DirectOutputCommitter, spark.storage.blockManagerTimeoutIntervalMs=300000, spark.databricks.driverNodeTypeId=Standard_E96ds_v5, spark.hadoop.fs.azure.skip.metrics=true, spark.hadoop.hive.hmshandler.retry.attempts=10, spark.databricks.eventLog.dir=eventlogs, spark.hadoop.fs.fcfs-wasb.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.hadoop.fs.gs.outputstream.upload.chunk.size=16777216, spark.hadoop.spark.driverproxy.customHeadersToProperties=X-Databricks-User-Token:spark.databricks.token,X-Databricks-Non-UC-User-Token:spark.databricks.non.uc.token,X-Databricks-Api-Url:spark.databricks.api.url,X-Databricks-ADLS-Gen1-Token:spark.databricks.adls.gen1.token,X-Databricks-ADLS-Gen2-Token:spark.databricks.adls.gen2.token,X-Databricks-Synapse-Token:spark.databricks.synapse.token,X-Databricks-AWS-Credentials:spark.databricks.aws.creds,X-Databricks-User-Id:spark.databricks.user.id,X-Databricks-User-Name:spark.databricks.user.name,X-Databricks-Oauth-Identity-Custom-Claim:spark.databricks.oauthCustomIdentityClaims, spark.databricks.automl.serviceEnabled=true, spark.databricks.clusterUsageTags.sparkEnvVarContainsDoubleQuotes=true, spark.databricks.credential.scope.fs.gs.auth.access.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeGCPTokenProvider, spark.akka.frameSize=256, spark.databricks.passthrough.s3a.tokenProviderClassName=com.databricks.backend.daemon.driver.aws.AwsCredentialContextTokenProvider, spark.databricks.delta.multiClusterWrites.enabled=true, spark.hadoop.fs.fcfs-s3n.impl.disable.cache=true, spark.databricks.clusterUsageTags.attribute_tag_dust_execution_env=, spark.hadoop.fs.AbstractFileSystem.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS, spark.serializer.objectStreamReset=100, spark.hadoop.fs.abfs.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.clusterUsageTags.sparkEnvVarContainsEscape=false, spark.sparkr.use.daemon=false, spark.databricks.clusterUsageTags.clusterFirstOnDemand=1, spark.files.useFetchCache=false, spark.sql.crossJoin.enabled=true, spark.sql.hive.metastore.version=0.13.0, spark.databricks.clusterUsageTags.driverPublicDns=20.5.66.227, spark.databricks.clusterUsageTags.clusterNumCustomTags=0, spark.databricks.clusterUsageTags.clusterName=Databricks-Purview, spark.sql.hive.metastore.sharedPrefixes=org.mariadb.jdbc,com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,microsoft.sql.DateTimeOffset,microsoft.sql.Types,com.databricks,com.codahale,com.fasterxml.jackson,shaded.databricks, spark.databricks.clusterUsageTags.clusterNumSshKeys=0, spark.hadoop.fs.s3a.attempts.maximum=10, spark.databricks.clusterUsageTags.orgId=8119356306712193, spark.memory.offHeap.enabled=true, spark.databricks.redactor=com.databricks.spark.util.DatabricksSparkLogRedactorProxy, spark.hadoop.fs.s3a.block.size=67108864, spark.hadoop.hive.server2.enable.doAs=false, spark.openlineage.transport.type=console, spark.databricks.clusterUsageTags.sparkEnvVarContainsDollarSign=false, spark.databricks.clusterUsageTags.clusterResourceClass=default, spark.hadoop.hive.server2.idle.session.timeout=900000, spark.hadoop.fs.abfs.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfss.impl.disable.cache=true, spark.databricks.clusterUsageTags.autoTerminationMinutes=120, spark.databricks.clusterUsageTags.attribute_tag_dust_suite=, spark.databricks.passthrough.glue.executorServiceFactoryClassName=com.databricks.backend.daemon.driver.credentials.GlueClientExecutorServiceFactory, spark.databricks.clusterUsageTags.enableDfAcls=false, spark.hadoop.parquet.block.size.row.check.max=10, spark.databricks.clusterUsageTags.driverInstancePrivateIp=10.60.28.15, spark.openlineage.version=v1, spark.databricks.clusterSource=UI, spark.openlineage.facets.disabled=[spark_unknown;], spark.hadoop.fs.cpfs-abfss.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.hadoop.fs.s3a.connection.timeout=50000, spark.hadoop.fs.cpfs-adl.impl.disable.cache=true, spark.hadoop.fs.fcfs-abfs.impl.disable.cache=true, spark.databricks.clusterUsageTags.region=australiaeast, spark.ui.port=40001, spark.sql.parquet.compression.codec=snappy, spark.databricks.workspaceUrl=adb-8119356306712193.13.azuredatabricks.net, spark.hadoop.fs.abfss.impl.disable.cache=true, spark.hadoop.parquet.filter.columnindex.enabled=false, spark.sql.execution.id=88, spark.databricks.passthrough.adls.gen2.tokenProviderClassName=com.databricks.backend.daemon.data.client.adl.AdlGen2CredentialContextTokenProvider, spark.hadoop.hive.hmshandler.retry.interval=2000, spark.hadoop.fs.cpfs-abfss.impl.disable.cache=true, spark.hadoop.fs.mlflowdbfs.impl=com.databricks.mlflowdbfs.MlflowdbfsFileSystem, spark.databricks.acl.client=com.databricks.spark.sql.acl.client.SparkSqlAclClient, spark.databricks.clusterUsageTags.dataPlaneRegion=australiaeast, spark.databricks.clusterUsageTags.clusterOwnerUserId=3661563703767194, spark.streaming.driver.writeAheadLog.allowBatching=true, spark.databricks.clusterUsageTags.enableLocalDiskEncryption=false, spark.hadoop.fs.azure.authorization.caching.enable=false, spark.databricks.credential.scope.fs.s3a.tokenProviderClassName=com.databricks.backend.daemon.driver.credentials.CredentialScopeS3TokenProvider, spark.hadoop.hive.server2.use.SSL=true, spark.hadoop.fs.s3a.threads.max=136, spark.repl.class.uri=spark://10.60.24.15:37693/classes, spark.sql.legacy.parquet.int96RebaseModeInWrite=LEGACY, spark.databricks.overrideDefaultCommitProtocol=org.apache.spark.sql.execution.datasources.SQLHadoopMapReduceCommitProtocol, spark.databricks.optimizer.optimizeLimit.maxLimit=1001, spark.driver.host=10.60.24.15, spark.driver.allowMultipleContexts=false, spark.databricks.driverNfs.enabled=true, spark.hadoop.parquet.page.metadata.validation.enabled=true, spark.hadoop.fs.fcfs-abfs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.hadoop.fs.s3a.multipart.size=10485760, spark.databricks.delta.logStore.crossCloud.fatal=true, spark.hadoop.fs.fcfs-s3a.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.task.reaper.enabled=true, spark.databricks.clusterUsageTags.clusterNodeType=Standard_E96ds_v5, spark.databricks.cloudProvider=Azure, spark.databricks.clusterUsageTags.instanceWorkerEnvId=workerenv-8119356306712193, spark.shuffle.memoryFraction=0.2, spark.databricks.clusterUsageTags.clusterScalingType=autoscaling, spark.hadoop.fs.fcfs-s3.impl.disable.cache=true, spark.hadoop.fs.dbfsartifacts.impl=com.databricks.backend.daemon.data.client.DBFSV1, spark.databricks.credential.redactor=com.databricks.logging.secrets.CredentialRedactorProxyImpl, spark.databricks.clusterUsageTags.isDpCpPrivateLinkEnabled=false, spark.app.name=Databricks Shell, spark.databricks.clusterUsageTags.clusterSku=STANDARD_SKU, spark.hadoop.fs.azure.cache.invalidator.type=com.databricks.encryption.utils.CacheInvalidatorImpl, spark.metrics.conf=/databricks/spark/conf/metrics.properties, spark.databricks.workload.name=JobRun, spark.sql.streaming.stopTimeout=15s, spark.hadoop.fs.s3a.assumed.role.credentials.provider=shaded.databricks.org.apache.hadoop.fs.s3a.DatabricksInstanceProfileCredentialsProvider, spark.delta.sharing.profile.provider.class=io.delta.sharing.DeltaSharingCredentialsProvider, spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.enabled=false, spark.files.overwrite=true, spark.databricks.tahoe.logStore.gcp.class=com.databricks.tahoe.store.GCPLogStore, spark.databricks.credential.scope.fs.impl=com.databricks.sql.acl.fs.CredentialScopeFileSystem, spark.hadoop.spark.databricks.io.parquet.verifyChecksumOnWrite.throwsException=false, spark.databricks.passthrough.s3a.threadPoolExecutor.factory.class=com.databricks.backend.daemon.driver.aws.S3APassthroughThreadPoolExecutorFactory, spark.hadoop.fs.s3a.retry.limit=6, spark.databricks.preemption.enabled=true, spark.hadoop.fs.gs.impl=shaded.databricks.com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemHadoop3, spark.hadoop.fs.dbfs.impl=com.databricks.backend.daemon.data.client.DbfsHadoop3, spark.databricks.clusterUsageTags.enableGlueCatalogCredentialPassthrough=false, spark.shuffle.reduceLocality.enabled=false, spark.databricks.repl.enableClassFileCleanup=true, spark.hadoop.fs.s3a.multipart.threshold=104857600, spark.hadoop.fs.fcfs-abfss.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.sparklyr-backend.threads=1, spark.hadoop.fs.fcfs-wasbs.impl.disable.cache=true, spark.databricks.clusterUsageTags.ignoreTerminationEventInAlerting=false, spark.hadoop.databricks.fs.perfMetrics.enable=true, spark.databricks.eventLog.enabled=true, spark.databricks.clusterUsageTags.managedResourceGroup=databricks-rg-ZEDBWBI02T-vrlfjwyjm5sky, spark.rdd.scope={"id":"spark_plan_2533","name":"Execute WriteIntoDeltaCommand"}, spark.hadoop.spark.hadoop.aws.glue.cache.table.size=1000, spark.databricks.api.url=https://australiaeast.azuredatabricks.net, spark.databricks.inherited.spark.databricks.api.url=https://australiaeast.azuredatabricks.net, spark.hadoop.fs.fcfs-wasbs.impl=com.databricks.sql.acl.fs.FixedCredentialsFileSystem, spark.databricks.clusterUsageTags.clusterPinned=false, spark.speculation.quantile=0.9, spark.databricks.job.id=922321804683480, spark.hadoop.fs.cpfs-s3a.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, spark.hadoop.fs.cpfs-adl.impl=com.databricks.sql.acl.fs.CredentialPassthroughFileSystem, userId=8162102493690770, spark.hadoop.fs.wasb.impl.disable.cache=true, spark.sql.legacy.createHiveTableByDefault=false, spark.openlineage.namespace=adb-8119356306712193.13.#0622-000257-omphh17r, spark.scheduler.mode=FAIR, spark.shuffle.service.port=4048, spark.databricks.clusterUsageTags.isIMv2Enabled=false, spark.hadoop.fs.fcfs-wasb.impl.disable.cache=true, spark.hadoop.databricks.s3.verifyBucketExists.enabled=false, spark.databricks.driverNfs.pathSuffix=.ephemeral_nfs, spark.sql.sources.commitProtocolClass=com.databricks.sql.transaction.directory.DirectoryAtomicCommitProtocol, spark.hadoop.fs.s3a.impl=com.databricks.common.filesystem.LokiFileSystem, spark.databricks.driverNfs.clusterWidePythonLibsEnabled=true, spark.scheduler.pool=7670323585580043917, spark.databricks.unityCatalog.credentialManager.tokenRefreshEnabled=true, spark.memory.offHeap.size=487180992512, spark.databricks.clusterUsageTags.azureSubscriptionId=f242a3b6-370c-4b2e-971d-a0399c53d6f0, spark.databricks.clusterUsageTags.clusterNoDriverDaemon=false, spark.databricks.clusterUsageTags.clusterMinWorkers=2, spark.databricks.clusterUsageTags.clusterState=Restarting, spark.sql.parquet.cacheMetadata=true, spark.databricks.clusterUsageTags.clusterSizeType=VM_CONTAINER, spark.databricks.clusterUsageTags.effectiveSparkVersion=12.1.x-scala2.12, spark.databricks.clusterUsageTags.driverContainerId=54143fa41c2a41a68bf7dc5182636b69, spark.worker.aioaLazyConfig.dbfsReadinessCheckClientClass=com.databricks.backend.daemon.driver.NephosDbfsReadinessCheckClient, spark.databricks.clusterUsageTags.clusterUnityCatalogMode=CUSTOM, spark.databricks.clusterUsageTags.clusterAllTags=[{"key":"Vendor","value":"Databricks"},{"key":"Creator","value":"chanh.tran@tafenswtest.edu.au"},{"key":"ClusterName","value":"Databricks-Purview"},{"key":"ClusterId","value":"0622-000257-omphh17r"},{"key":"CostCentre","value":"CTP/180529-01"},{"key":"Environment","value":"Test"},{"key":"Owner","value":"Enterprise Data Services"},{"key":"ServiceName","value":"TRIP"},{"key":"DatabricksEnvironment","value":"workerenv-8119356306712193"}], spark.hadoop.parquet.page.write-checksum.enabled=true, spark.cleaner.referenceTracking.blocking=false, spark.hadoop.databricks.dbfs.client.version=v2, spark.hadoop.fs.fcfs-s3a.impl.disable.cache=true, spark.databricks.clusterUsageTags.attribute_tag_service=, spark.databricks.cloudfetch.requesterClassName=com.databricks.spark.sql.cloudfetch.DataDaemonCloudPresignedUrlRequester, spark.databricks.clusterUsageTags.clusterOwnerOrgId=8119356306712193, spark.jobGroup.id=7670323585580043917_8469927463799253158_job-922321804683480-run-5532172-action-5363477929991621, spark.databricks.workload.id=8119356306712193-922321804683480-5532172, spark.databricks.service.dbutils.server.backend=com.databricks.dbconnect.SparkServerDBUtils, spark.hadoop.fs.s3a.impl.disable.cache=true, spark.databricks.clusterUsageTags.sparkEnvVarContainsBacktick=false, spark.databricks.credential.aws.secretKey.redactor=com.databricks.spark.util.AWSSecretKeyRedactorProxy}) by listener OpenLineageSparkListener took 10.5513922s. 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 88 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:03.839Z","run":{"runId":"f291971f-4742-4b57-89c6-43d0652fbbfc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, groupid, userid, timeadded, component, itemid, _DLRawZoneTimeStamp, year, month, day]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"}],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:48:34.707346","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"06","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"28","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:49:13 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:49:13 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:49:13 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:49:03.943Z","run":{"runId":"f291971f-4742-4b57-89c6-43d0652fbbfc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, groupid, userid, timeadded, component, itemid, _DLRawZoneTimeStamp, year, month, day]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"}],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:48:34.707346","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"06","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"28","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532172","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/raw/load-raw-deltatable","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"922321804683480","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 88 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:03.96Z","run":{"runId":"f291971f-4742-4b57-89c6-43d0652fbbfc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, groupid, userid, timeadded, component, itemid, _DLRawZoneTimeStamp, year, month, day]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"}],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:48:34.707346","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"06","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"28","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 90 23/06/28 07:49:13 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 91 23/06/28 07:49:13 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 92 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 92 23/06/28 07:49:13 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:05.112Z","run":{"runId":"b8f3c29a-a9e5-4222-b35e-aba23455864b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":161,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.CaseWhen","num-children":3,"branches":[{"product-class":"scala.Tuple2","_1":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"_2":[{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]}],"elseValue":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":150,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.CaseWhen","num-children":3,"branches":[{"product-class":"scala.Tuple2","_1":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"_2":[{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]}],"elseValue":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Length","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Concat","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"day","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":139,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.DayOfMonth","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"month","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":129,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Month","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"year","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":120,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Year","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"date","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLRawZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":112,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.FromUTCTimestamp","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"timestamp","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 07:48:34.707346","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"Australia/Sydney","dataType":"string"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":100,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":88,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":101,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":89,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":102,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":90,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":103,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":91,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":104,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":92,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":105,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":93,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":88,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":89,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":90,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":91,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":92,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":93,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":81,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":81,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":74,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":67,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":60,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":53,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":46,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":33,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"id","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":33,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"groupid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":34,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"userid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":35,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"timeadded","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":36,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":37,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"itemid","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":38,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"long","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":10,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":11,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":12,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":13,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":14,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":15,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_lmsuu_mdl_groups_members_delta","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"}]}},"inputFacets":{}}],"outputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"long"},{"name":"groupid","type":"long"},{"name":"userid","type":"long"},{"name":"timeadded","type":"long"},{"name":"component","type":"string"},{"name":"itemid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"year","type":"string"},{"name":"month","type":"string"},{"name":"day","type":"string"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"id"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"groupid"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"userid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"timeadded"}]},"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"component"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/json/year=2023/month=06/day=28/lmsuu.mdl_groups_members_2023-06-28_174631_940.json.gz","field":"itemid"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:49:13 INFO HiveMetaStore: 2: get_database: raw 23/06/28 07:49:13 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:49:13 INFO HiveMetaStore: 2: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:49:13 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:49:13 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:49:13 INFO HiveMetaStore: 2: get_database: raw 23/06/28 07:49:13 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:49:13 INFO AbstractDatabricksHandler: dbfs:/user/hive/warehouse/raw.db/lms_lmsuu_mdl_groups_members 23/06/28 07:49:13 WARN InputFieldsCollector: Could not extract dataset identifier from org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier 23/06/28 07:49:13 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:49:05.273Z","run":{"runId":"3ae089c6-69ef-4ba9-a510-6ce48d0a906c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.CreateTable","num-children":1,"name":0,"tableSchema":[],"partitioning":[],"tableSpec":null,"ignoreIfExists":true},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier","num-children":0,"catalog":null,"identifier":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_create_table_raw_db_lms_lmsuu_mdl_groups_members","facets":{}},"inputs":[],"outputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/raw.db/lms_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[]},"storage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/StorageDatasetFacet.json#/$defs/StorageDatasetFacet","storageLayer":"unity","fileFormat":"parquet"},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse/raw.db","name":"raw.lms_lmsuu_mdl_groups_members","type":"TABLE"}]},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"CREATE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 94 23/06/28 07:49:13 INFO SQLExecution: Received onJobStart for query with unregistered executionId 94 23/06/28 07:49:13 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:49:54 INFO DataSourceFactory$: DataSource Jdbc URL: jdbc:mariadb://consolidated-australiaeast-prod-metastore-addl-1.mysql.database.azure.com:3306/organization8119356306712193?useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertificate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt 23/06/28 07:49:54 INFO HikariDataSource: metastore-monitor - Starting... 23/06/28 07:49:54 INFO HikariDataSource: metastore-monitor - Start completed. 23/06/28 07:49:54 INFO HikariDataSource: metastore-monitor - Shutdown initiated... 23/06/28 07:49:54 INFO HikariDataSource: metastore-monitor - Shutdown completed. 23/06/28 07:49:54 INFO MetastoreMonitor: Metastore healthcheck successful (connection duration = 321 milliseconds) 23/06/28 07:50:03 INFO HiveMetaStore: 3: get_database: default 23/06/28 07:50:03 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:50:03 INFO HiveMetaStore: 3: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:50:03 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:50:03 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:50:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:50:03 INFO DriverCorral: Metastore health check ok 23/06/28 07:50:13 INFO DriverCorral: DBFS health check ok 23/06/28 07:50:41 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:50:41 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:50:41 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:50:41 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:50:41 INFO DriverCorral: Starting python repl ReplId-d8661-bce1f-96ec4 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:95. Current active queries:1 23/06/28 07:50:41 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$ 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:95. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:96. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:96. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:97. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:50:41.321Z","run":{"runId":"ff5bd043-1dc1-4676-b70c-d4858f4461bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:97. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.321Z","run":{"runId":"ff5bd043-1dc1-4676-b70c-d4858f4461bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:98. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:98. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:99. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:99. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:100. Current active queries:1 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.327Z","run":{"runId":"f9b2ce90-8b75-4a20-9756-a00f9352221c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:100. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:101. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:101. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:102. Current active queries:1 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.331Z","run":{"runId":"09add5f4-603c-4d3b-8941-d3d93c0d3306","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:102. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:103. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:103. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:104. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.335Z","run":{"runId":"21bda84d-a62a-422f-a50e-c12a5ed1abee","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:104. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:105. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:105. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:106. Current active queries:1 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.339Z","run":{"runId":"b783b76f-391e-462f-b5b5-2b26065ab7dc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:106. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:107. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:107. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:108. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.343Z","run":{"runId":"1457b6fe-dd63-46bc-b592-f5192fb1b4ab","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:108. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:109. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:109. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.347Z","run":{"runId":"230dfe46-d86d-4b58-996b-36ce66e68fe4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:110. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:110. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:111. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:111. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.351Z","run":{"runId":"c3c7a306-4d5d-4263-be40-489c8ac5216a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:112. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:112. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:113. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:113. Current active queries:0 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.355Z","run":{"runId":"5ee31c06-35ec-41c0-b615-52205d89f81f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:114. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:114. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:115. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:115. Current active queries:0 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.36Z","run":{"runId":"cbd57324-3ece-4235-aac2-449079f978a7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:116. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:116. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.365Z","run":{"runId":"a39481f1-7a63-4da7-9780-38d7f9693cab","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:117. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:117. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:118. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.37Z","run":{"runId":"0d5ecd97-d4da-4093-983f-992520f4c9b6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:118. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:119. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:119. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:120. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.374Z","run":{"runId":"7df25ab6-7697-43b8-bf83-f0414de3a529","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:120. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:121. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.38Z","run":{"runId":"8dcc8c5e-823f-4480-ab56-3c5b184ba44f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:121. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:122. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.386Z","run":{"runId":"279be569-4d66-4c34-8a8d-3fa3ab50f748","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:122. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:123. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:123. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:124. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.391Z","run":{"runId":"2f13d30c-b633-439f-90bf-764c3cc43742","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:124. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:125. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:125. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:126. Current active queries:1 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.396Z","run":{"runId":"f35ef02e-46dd-4334-88f0-253409bc2604","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:126. Current active queries:0 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:127. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:127. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.403Z","run":{"runId":"f6870cc9-1c23-47d8-ac34-4011506af095","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO ClusterLoadMonitor: Added query with execution ID:128. Current active queries:1 23/06/28 07:50:41 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:41 INFO ClusterLoadMonitor: Removed query with execution ID:128. Current active queries:0 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.408Z","run":{"runId":"aadc2e6f-9d45-4c6a-8715-f0bfdf7b164c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO JupyterDriverLocal: Starting gateway server for repl ReplId-d8661-bce1f-96ec4 23/06/28 07:50:41 INFO PythonPy4JUtil: Using pinned thread mode in Py4J 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.411Z","run":{"runId":"5b9f966f-6b71-4e42-b777-83eabeeb9198","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.415Z","run":{"runId":"e2a4c5dc-dbf8-4ed0-ab78-bb6b9b28e18f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.419Z","run":{"runId":"386a6e72-3ff6-42e9-a551-aa3194767377","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.427Z","run":{"runId":"05fad59b-ad6c-46c8-a564-f4238b695967","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.432Z","run":{"runId":"08bbfa02-2382-4637-b177-20c3a15f0585","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.436Z","run":{"runId":"b4cef90f-0463-45e3-b8a9-7c2e1fa0e7d6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.441Z","run":{"runId":"3d4a7922-38f1-4ae3-ad1b-c51e576edfe6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.451Z","run":{"runId":"23baa80e-46be-48e9-8f2b-76ecdfea1ef7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.454Z","run":{"runId":"472549bc-ded2-4834-a337-9017ba9b2b00","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.458Z","run":{"runId":"546e7fe3-6ab6-419a-9818-44e2dd625dcf","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.462Z","run":{"runId":"ac903921-9587-4c9e-86ba-8e0c3bb5fb5e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.466Z","run":{"runId":"85f022f3-95a5-47c7-8c85-43ea391b1c8b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.47Z","run":{"runId":"00ec35af-8b6b-488e-87af-89411c007a77","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.477Z","run":{"runId":"d03be767-8d8d-448d-99a6-491459d94782","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:41 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:41.481Z","run":{"runId":"042bd3bc-b96c-418e-8496-47b9ee4775bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:41 INFO VirtualenvCloneHelper: Creating notebook-scoped virtualenv for d25a10dc-7b9d-45a9-a230-9bf5986b4e0d 23/06/28 07:50:41 INFO Utils: resolved command to be run: List(virtualenv, /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d, -p, /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, --no-download, --no-setuptools, --no-wheel) 23/06/28 07:50:42 INFO PythonEnvCloneHelper$: Created python virtualenv: /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d 23/06/28 07:50:42 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, -c, import sys; dirs=[p for p in sys.path if 'package' in p]; print(' '.join(dirs))) 23/06/28 07:50:42 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d/bin/python, -c, from distutils.sysconfig import get_python_lib; print(get_python_lib())) 23/06/28 07:50:42 INFO PythonEnvCloneHelper$: Created sites.pth at /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d/lib/python3.9/site-packages/sites.pth 23/06/28 07:50:42 INFO NotebookScopedPythonEnvManager: Time spent to start virtualenv /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d is 219(ms) 23/06/28 07:50:42 INFO NotebookScopedPythonEnvManager: Registered /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d/lib/python3.9/site-packages with the WatchService sun.nio.fs.LinuxWatchService$LinuxWatchKey@7bade44d 23/06/28 07:50:42 INFO IpykernelUtils$: Python process builder: [bash, /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d/python_start_d25a10dc-7b9d-45a9-a230-9bf5986b4e0d.sh, /databricks/spark/python/pyspark/wrapped_python.py, root, /local_disk0/.ephemeral_nfs/envs/pythonEnv-d25a10dc-7b9d-45a9-a230-9bf5986b4e0d/bin/python, /databricks/python_shell/scripts/db_ipykernel_launcher.py, -f, /databricks/kernel-connections/6cfc27919b08cdcd807967efac7b2ef36bc023358d9ac20aaf98f08ebac3ea21.json] 23/06/28 07:50:42 INFO IpykernelUtils$: Cgroup isolation disabled, not placing python process in repl cgroup 23/06/28 07:50:43 INFO PythonDriverWrapper: setupRepl:ReplId-d8661-bce1f-96ec4: finished to load 23/06/28 07:50:43 INFO DriverCorral: Starting scala repl ReplId-6b16f-c615f-2e9ee-1 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:129. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:129. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:130. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:130. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:131. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:131. Current active queries:0 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:50:43.443Z","run":{"runId":"f50e78f6-07db-49d4-8005-f7e4e78fb8e3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:132. Current active queries:1 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.443Z","run":{"runId":"f50e78f6-07db-49d4-8005-f7e4e78fb8e3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:132. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:133. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:133. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:134. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.448Z","run":{"runId":"c48f51f9-607f-4e4a-a1a6-7881883be7e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:134. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:135. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:135. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:136. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.452Z","run":{"runId":"4f814e9a-6cbd-4045-9615-df2739b6d86e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:136. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:137. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:137. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:138. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.457Z","run":{"runId":"fb5bedac-194b-4741-9378-9c7711b797c7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:138. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:139. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:139. Current active queries:0 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.46Z","run":{"runId":"d81cca58-293c-47c3-a513-fa70d62c64ce","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:140. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:140. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:141. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:141. Current active queries:0 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.464Z","run":{"runId":"217cd8ec-6757-48b7-ac39-c3943c0b1103","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:142. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:142. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:143. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:143. Current active queries:0 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.468Z","run":{"runId":"9a5573e7-f139-4f18-a2b8-f3d6ea43c2c0","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:144. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:144. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:145. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:145. Current active queries:0 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.472Z","run":{"runId":"b84f0be4-fce6-46d0-8507-02ed118ccb9c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:146. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:146. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:147. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:147. Current active queries:0 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.476Z","run":{"runId":"bb854705-9574-4312-aebf-f81ca44d0ef3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:148. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:148. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:149. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:149. Current active queries:0 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.481Z","run":{"runId":"871763d8-279a-459e-aaa3-82472a0b8d8b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:150. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:150. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:151. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:151. Current active queries:0 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.486Z","run":{"runId":"3e3f7ab5-79ea-4438-a576-d563dd4e8ceb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:152. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:152. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:153. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.49Z","run":{"runId":"64c0eede-ca6f-44d9-a682-7849043e4ad9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:153. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:154. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:154. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:155. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.494Z","run":{"runId":"30ae56bd-217f-4167-a701-e6d67722b9de","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:155. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:156. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:156. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:157. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.498Z","run":{"runId":"c3aee67f-dfde-4c68-83a0-c5a844714df1","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:157. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:158. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:158. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:159. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.501Z","run":{"runId":"8df8fc35-002b-437c-8815-fd6e411405ad","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:159. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:160. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:160. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:161. Current active queries:1 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.506Z","run":{"runId":"17ff59ce-a912-4391-b0fc-8f35a2194520","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:161. Current active queries:0 23/06/28 07:50:43 INFO ClusterLoadMonitor: Added query with execution ID:162. Current active queries:1 23/06/28 07:50:43 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:50:43 INFO ClusterLoadMonitor: Removed query with execution ID:162. Current active queries:0 23/06/28 07:50:43 INFO ScalaDriverWrapper: setupRepl:ReplId-6b16f-c615f-2e9ee-1: finished to load 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.51Z","run":{"runId":"6c2d78a4-7eb4-46a2-9ce8-dea1e6b7f8a9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6466893675859450898_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.515Z","run":{"runId":"f82da6db-2efe-4aa8-b513-64dd7419246b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6466893675859450898_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.52Z","run":{"runId":"7b6c5fd6-052f-46e0-bb79-52ea9dcb2bb6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.526Z","run":{"runId":"7f860c77-fb47-4987-8c33-71acf8f5d2b5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.531Z","run":{"runId":"5c603c22-06e9-4f8e-9c99-c37730eb88a4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6466893675859450898_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7584386408925248858_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.535Z","run":{"runId":"9ef64ad5-74d2-431f-9e68-b4c565d099ca","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7584386408925248858_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.54Z","run":{"runId":"5b7d6761-d8eb-4ae0-896e-7f23a51828a9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.545Z","run":{"runId":"c79180c1-9805-4326-8c21-bfa04ed43806","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.55Z","run":{"runId":"0bb758f5-ca2f-430b-bf9d-7cbde33c8061","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.555Z","run":{"runId":"f0b03b31-29b9-42f3-88ac-845aff854333","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.561Z","run":{"runId":"f2367260-c4c9-4a54-b7c2-b6060385b4aa","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.567Z","run":{"runId":"601de7fb-4118-411a-a514-bfbaef3172db","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.571Z","run":{"runId":"2ab0a3b1-d24d-4043-a82f-2ab25562e53a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.575Z","run":{"runId":"bdf8b8ef-4d83-4d21-860b-08ac3b9ccd89","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.578Z","run":{"runId":"e1272382-bf8b-4c3c-b7a5-1120690ed37b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.583Z","run":{"runId":"1b47ecb9-e8ab-48d7-9cea-3b9dac1f7038","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.587Z","run":{"runId":"f50fb3da-21ad-451b-8d2d-b8dd4d0706d5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7584386408925248858_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5917361165411512234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:50:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:50:43.59Z","run":{"runId":"7515d8ce-45b2-41e1-836b-08fa7c4b9f84","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:50:43 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5917361165411512234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5917361165411512234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5685537677960990360_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5685537677960990360_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5685537677960990360_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8402512417420200972_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:43 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8402512417420200972_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8402512417420200972_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6910431117469587746_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6910431117469587746_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6910431117469587746_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9156229478496554264_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9156229478496554264_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInRead' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInRead' instead. 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9156229478496554264_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5945687099119027456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5945687099119027456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInWrite' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInWrite' instead. 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5945687099119027456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7928692569584000649_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7928692569584000649_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7928692569584000649_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8582775818591055295_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8582775818591055295_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8582775818591055295_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7310174771729813618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:44 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7310174771729813618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:45 INFO DriverILoop: Set class prefix to: $line03c6726cdf344982a725e349c463a644 23/06/28 07:50:45 INFO DriverILoop: set ContextClassLoader 23/06/28 07:50:45 INFO DriverILoop: initialized intp 23/06/28 07:50:46 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7310174771729813618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:46 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6607440765819715504_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:46 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6607440765819715504_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:46 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6607440765819715504_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:46 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8940069533243381533_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:46 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8940069533243381533_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8940069533243381533_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8528454694191126279_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8528454694191126279_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8528454694191126279_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4980761059013942591_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4980761059013942591_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4980761059013942591_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6796071206574688499_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6796071206574688499_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6796071206574688499_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9151120022556171268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9151120022556171268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9151120022556171268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8666987738199903524_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8666987738199903524_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8666987738199903524_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8467740202674821765_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8467740202674821765_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8467740202674821765_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8754833515635018493_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8754833515635018493_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8754833515635018493_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5818099539888924323_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5818099539888924323_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5818099539888924323_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6633725481549499361_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6633725481549499361_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6633725481549499361_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8621777467837744472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:47 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8621777467837744472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8621777467837744472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8242708374914741268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8242708374914741268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8242708374914741268_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6559848027845371075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6559848027845371075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6559848027845371075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7610176484700711065_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7610176484700711065_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7610176484700711065_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5482216858478419830_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5482216858478419830_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5482216858478419830_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8104086130341107779_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8104086130341107779_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8104086130341107779_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9073674291918703947_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9073674291918703947_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9073674291918703947_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8938028022749866293_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8938028022749866293_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8938028022749866293_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6491730259621933391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6491730259621933391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6491730259621933391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8545883797038301664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8545883797038301664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8545883797038301664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8248716619423058567_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:48 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8248716619423058567_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8248716619423058567_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6490080236482508255_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6490080236482508255_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6490080236482508255_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4618532268778825258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4618532268778825258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4618532268778825258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5947314254574643091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5947314254574643091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5947314254574643091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6284048241614433930_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6284048241614433930_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6284048241614433930_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7580080439846139884_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7580080439846139884_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7580080439846139884_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7712661594501571842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7712661594501571842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7712661594501571842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5215218569351117781_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5215218569351117781_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5215218569351117781_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4780819260296757554_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_4780819260296757554_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4780819260296757554_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8459841352994888816_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8459841352994888816_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8459841352994888816_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5464474863346968647_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:49 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5464474863346968647_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5464474863346968647_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8224835914039345660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8224835914039345660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8224835914039345660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6065264772325998639_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6065264772325998639_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6065264772325998639_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4936186005154805674_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4936186005154805674_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4936186005154805674_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5458304722868829288_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5458304722868829288_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5458304722868829288_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7967198908056397209_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7967198908056397209_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7967198908056397209_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5711202643269309258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5711202643269309258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5711202643269309258_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6067527025565441901_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6067527025565441901_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6067527025565441901_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5440731814836279890_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5440731814836279890_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5440731814836279890_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9206200959334681426_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9206200959334681426_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9206200959334681426_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6724780672256218900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:50 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6724780672256218900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6724780672256218900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6632647468734175636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6632647468734175636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6632647468734175636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7533871395054714506_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7533871395054714506_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7533871395054714506_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6868006272598470012_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6868006272598470012_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6868006272598470012_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5522739752806541767_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5522739752806541767_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5522739752806541767_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7749067986254103640_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7749067986254103640_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7749067986254103640_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6945018877241013602_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6945018877241013602_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6945018877241013602_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6384053372104723790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6384053372104723790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6384053372104723790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8722897240285026365_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8722897240285026365_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8722897240285026365_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7666588004076351283_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7666588004076351283_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7666588004076351283_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6950010231334516696_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:51 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6950010231334516696_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6950010231334516696_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6374976250211116429_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6374976250211116429_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6374976250211116429_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8447513855109438760_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8447513855109438760_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8447513855109438760_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7291012679210515011_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7291012679210515011_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7291012679210515011_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8819618070671668019_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8819618070671668019_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8819618070671668019_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7829875560384131515_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7829875560384131515_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7829875560384131515_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8482086373318091084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8482086373318091084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8482086373318091084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7393023149232417384_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7393023149232417384_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7393023149232417384_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5266454882882429327_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5266454882882429327_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5266454882882429327_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7003027805602847680_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7003027805602847680_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7003027805602847680_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9006752975881183249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:52 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9006752975881183249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9006752975881183249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6947137693875137556_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6947137693875137556_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6947137693875137556_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9014207804405488412_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9014207804405488412_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9014207804405488412_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5925718742870583336_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5925718742870583336_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5925718742870583336_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8741619031004220178_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8741619031004220178_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8741619031004220178_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5736505629525418560_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5736505629525418560_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5736505629525418560_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6293231402492642892_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6293231402492642892_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6293231402492642892_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5610710992340303170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5610710992340303170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5610710992340303170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5142177133673207170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5142177133673207170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5142177133673207170_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6885239288879079358_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6885239288879079358_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6885239288879079358_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6720219309893126091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:53 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6720219309893126091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6720219309893126091_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6625122361726904418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6625122361726904418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6625122361726904418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7393768038571765805_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7393768038571765805_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7393768038571765805_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6826908018319769320_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6826908018319769320_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6826908018319769320_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7729425315296072867_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7729425315296072867_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7729425315296072867_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7748975354777420397_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7748975354777420397_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7748975354777420397_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9007276298199828428_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9007276298199828428_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9007276298199828428_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5272057002725156379_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5272057002725156379_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5272057002725156379_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8821676626054094308_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8821676626054094308_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8821676626054094308_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8516793323343796142_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8516793323343796142_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8516793323343796142_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4882311311387128231_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:54 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4882311311387128231_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4882311311387128231_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5666062252140427531_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5666062252140427531_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5666062252140427531_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9090795617378563608_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9090795617378563608_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9090795617378563608_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7738258984789761705_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7738258984789761705_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7738258984789761705_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6790365060706505971_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6790365060706505971_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6790365060706505971_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7585243197176671842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7585243197176671842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7585243197176671842_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5514706636850647864_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5514706636850647864_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5514706636850647864_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5333892780805965086_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5333892780805965086_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5333892780805965086_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8924063908873515354_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8924063908873515354_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8924063908873515354_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6948666307969204661_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6948666307969204661_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6948666307969204661_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:55 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6567814739148762396_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6567814739148762396_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6567814739148762396_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5163719118693479618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5163719118693479618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5163719118693479618_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8416380903359899487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8416380903359899487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8416380903359899487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8380730833620981783_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8380730833620981783_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8380730833620981783_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5924392634858990725_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5924392634858990725_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5924392634858990725_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8842652888431546745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8842652888431546745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8842652888431546745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5828612516013715546_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5828612516013715546_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5828612516013715546_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6189558078892703388_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6189558078892703388_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6189558078892703388_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7649110685066495277_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7649110685066495277_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7649110685066495277_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9017250783267856940_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9017250783267856940_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9017250783267856940_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:56 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4664266333058716245_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4664266333058716245_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4664266333058716245_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8701553839824133177_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8701553839824133177_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8701553839824133177_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7572694955098089609_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7572694955098089609_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7572694955098089609_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7528437331744427341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7528437331744427341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7528437331744427341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7585286418923887558_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7585286418923887558_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7585286418923887558_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8773010654645589302_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8773010654645589302_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8773010654645589302_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5024679777143275790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5024679777143275790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5024679777143275790_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6436281874185804610_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6436281874185804610_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6436281874185804610_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4652260586196466018_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4652260586196466018_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4652260586196466018_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8895750724401104312_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:57 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8895750724401104312_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8895750724401104312_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8494344888874045550_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8494344888874045550_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8494344888874045550_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8471016330678612473_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8471016330678612473_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8471016330678612473_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8316946985749303120_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8316946985749303120_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8316946985749303120_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8655093642540686180_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8655093642540686180_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8655093642540686180_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8665208084717078751_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8665208084717078751_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8665208084717078751_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6978469870847152498_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6978469870847152498_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6978469870847152498_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6409108200314108389_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6409108200314108389_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6409108200314108389_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7506720216949078162_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7506720216949078162_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7506720216949078162_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8390020992196276487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8390020992196276487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8390020992196276487_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8414336953994696106_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:58 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8414336953994696106_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8414336953994696106_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7738943578728967852_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7738943578728967852_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7738943578728967852_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8449934550631183064_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8449934550631183064_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8449934550631183064_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8273884047769280827_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8273884047769280827_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8273884047769280827_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7570126064021671893_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7570126064021671893_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7570126064021671893_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7120639799269613536_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7120639799269613536_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7120639799269613536_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7847557479170052221_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7847557479170052221_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7847557479170052221_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7380117854163586636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7380117854163586636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7380117854163586636_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8562685601996821055_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8562685601996821055_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8562685601996821055_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8915988578856799988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8915988578856799988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8915988578856799988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8146150095213898040_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:50:59 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8146150095213898040_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8146150095213898040_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9149771571356321660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9149771571356321660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9149771571356321660_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7583544441359497115_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7583544441359497115_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7583544441359497115_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7589134462632763490_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7589134462632763490_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7589134462632763490_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5475562446496077694_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5475562446496077694_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5475562446496077694_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6173000762236366642_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6173000762236366642_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6173000762236366642_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5694254439266370432_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5694254439266370432_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5694254439266370432_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5532256201833862871_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5532256201833862871_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5532256201833862871_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7647247967053559039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7647247967053559039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7647247967053559039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8933982636430301075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_8933982636430301075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8933982636430301075_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6949514959987476828_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:00 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6949514959987476828_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6949514959987476828_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4752574302534009919_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4752574302534009919_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4752574302534009919_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6907340724912701818_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6907340724912701818_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6907340724912701818_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7423426228633958586_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7423426228633958586_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7423426228633958586_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7282226623528871664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7282226623528871664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7282226623528871664_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5759024687525850405_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5759024687525850405_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5759024687525850405_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7048540849629401080_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7048540849629401080_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7048540849629401080_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5499519101415467043_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5499519101415467043_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5499519101415467043_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6430855420814369675_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6430855420814369675_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6430855420814369675_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7695464497310621988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7695464497310621988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7695464497310621988_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8300727151522388084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:01 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8300727151522388084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8300727151522388084_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8479213179076509097_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8479213179076509097_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8479213179076509097_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6667152507046532900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6667152507046532900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6667152507046532900_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5694987464217225561_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5694987464217225561_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5694987464217225561_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9003305096559738902_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9003305096559738902_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9003305096559738902_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5235285068153846979_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5235285068153846979_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5235285068153846979_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8920833063099184037_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8920833063099184037_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8920833063099184037_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5863647999604363486_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5863647999604363486_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5863647999604363486_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4670744296149107949_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4670744296149107949_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4670744296149107949_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7970527547894842651_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7970527547894842651_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7970527547894842651_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8346699410357369234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:02 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8346699410357369234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8346699410357369234_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5481744182013212446_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5481744182013212446_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5481744182013212446_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6354371059688869876_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6354371059688869876_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6354371059688869876_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6503460841981283335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6503460841981283335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6503460841981283335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7818649140332604073_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7818649140332604073_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7818649140332604073_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5670846991786355711_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_5670846991786355711_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5670846991786355711_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4853005466201208835_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_4853005466201208835_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4853005466201208835_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5449427171424089391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_5449427171424089391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5449427171424089391_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7194145428263167534_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7194145428263167534_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7194145428263167534_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6796154917811480481_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6796154917811480481_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6796154917811480481_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7679243895633525005_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:03 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7679243895633525005_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7679243895633525005_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8108789453668731941_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_8108789453668731941_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8108789453668731941_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8111318122890403466_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_8111318122890403466_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8111318122890403466_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6597261844255362371_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6597261844255362371_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6597261844255362371_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8653432109261995335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_8653432109261995335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8653432109261995335_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6230877060554834341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6230877060554834341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6230877060554834341_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6514574484033615112_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6514574484033615112_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6514574484033615112_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6832637343642311410_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6832637343642311410_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6832637343642311410_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6859884576445656378_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:04 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6859884576445656378_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6859884576445656378_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7486789507075075943_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_7486789507075075943_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7486789507075075943_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5803657966189765910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_5803657966189765910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5803657966189765910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6391247574413452880_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6391247574413452880_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6391247574413452880_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5838085196427426317_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_5838085196427426317_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5838085196427426317_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5724893865621001423_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5724893865621001423_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5724893865621001423_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6244878711801244562_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_6244878711801244562_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6244878711801244562_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6546098216991807541_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6546098216991807541_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6546098216991807541_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5546340279467462836_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5546340279467462836_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5546340279467462836_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5141876969720092344_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5141876969720092344_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5141876969720092344_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7796826693871100886_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:05 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7796826693871100886_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7796826693871100886_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6896514698161571472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6896514698161571472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6896514698161571472_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8960312596204687408_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8960312596204687408_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8960312596204687408_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7994384921478647645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7994384921478647645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7994384921478647645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6302234313581403202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6302234313581403202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6302234313581403202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8066173403090028372_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8066173403090028372_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8066173403090028372_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6929707855116157961_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6929707855116157961_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6929707855116157961_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7718254997771136728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7718254997771136728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7718254997771136728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5911783217273116063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5911783217273116063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5911783217273116063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6752004874713692063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6752004874713692063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6752004874713692063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5977938592862938548_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:06 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5977938592862938548_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5977938592862938548_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8948094593504259103_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8948094593504259103_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8948094593504259103_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5902588485032180176_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5902588485032180176_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5902588485032180176_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5894033445806632338_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5894033445806632338_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5894033445806632338_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5205293777461957461_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5205293777461957461_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5205293777461957461_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8099298818536923829_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8099298818536923829_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8099298818536923829_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7747455268006517196_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7747455268006517196_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7747455268006517196_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8329948080587492768_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8329948080587492768_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8329948080587492768_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5938519851856698888_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5938519851856698888_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5938519851856698888_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5234428343165777259_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5234428343165777259_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5234428343165777259_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5341142721002704501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:07 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5341142721002704501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5341142721002704501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5939109874902525897_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5939109874902525897_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5939109874902525897_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6422670996860353981_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6422670996860353981_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6422670996860353981_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8010569274355437249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8010569274355437249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8010569274355437249_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6929030128934094628_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6929030128934094628_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6929030128934094628_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4947930481970568551_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4947930481970568551_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4947930481970568551_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5914639094985679800_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5914639094985679800_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5914639094985679800_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5086086522444125368_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5086086522444125368_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5086086522444125368_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9112943993438653395_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9112943993438653395_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9112943993438653395_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6437419015800826225_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6437419015800826225_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6437419015800826225_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9010815843492184614_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:08 INFO ProgressReporter$: Added result fetcher for 7716632506684841697_9010815843492184614_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9010815843492184614_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7667001846492416976_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7667001846492416976_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7667001846492416976_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7889265182097082039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7889265182097082039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7889265182097082039_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8237627705801518665_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8237627705801518665_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8237627705801518665_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9195974931270774049_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9195974931270774049_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9195974931270774049_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8724185942922530758_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8724185942922530758_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8724185942922530758_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5287726018980381006_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5287726018980381006_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5287726018980381006_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7508456819629665456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7508456819629665456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7508456819629665456_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8897738617454970547_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8897738617454970547_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8897738617454970547_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7910609329952753425_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7910609329952753425_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7910609329952753425_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6646675521576010905_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:09 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6646675521576010905_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6646675521576010905_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8562205774420374450_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8562205774420374450_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8562205774420374450_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6731846706757894445_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6731846706757894445_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6731846706757894445_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4726905608066576535_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4726905608066576535_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4726905608066576535_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7192868917407641491_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7192868917407641491_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7192868917407641491_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6407997134576996742_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6407997134576996742_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6407997134576996742_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6691854881321555745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6691854881321555745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6691854881321555745_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6649932256101310497_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6649932256101310497_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6649932256101310497_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7109053087867424542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7109053087867424542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7109053087867424542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8145731836431822910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8145731836431822910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8145731836431822910_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9152595938379136072_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:10 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9152595938379136072_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9152595938379136072_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7969291522684989202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7969291522684989202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7969291522684989202_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5879364582188452418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5879364582188452418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5879364582188452418_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9058764093145358063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9058764093145358063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9058764093145358063_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8822674989482293247_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8822674989482293247_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8822674989482293247_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6085467588926419728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6085467588926419728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6085467588926419728_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5882098183772053167_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5882098183772053167_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5882098183772053167_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8464273812960867726_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8464273812960867726_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8464273812960867726_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8615417209638213109_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8615417209638213109_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8615417209638213109_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8103739188796947508_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8103739188796947508_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8103739188796947508_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6881446516832375501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:11 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6881446516832375501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6881446516832375501_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4821366044220184932_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4821366044220184932_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4821366044220184932_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_9065458156017812477_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_9065458156017812477_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_9065458156017812477_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4675852258446896695_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4675852258446896695_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4675852258446896695_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5701353843793306750_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5701353843793306750_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5701353843793306750_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5312634717656402093_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5312634717656402093_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5312634717656402093_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4798039539073935881_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4798039539073935881_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4798039539073935881_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8129206777291908645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8129206777291908645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8129206777291908645_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6921715797278802449_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6921715797278802449_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6921715797278802449_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_8951450409777269542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_8951450409777269542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_8951450409777269542_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5081905987464520300_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:12 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5081905987464520300_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5081905987464520300_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_7836001018745200024_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Added result fetcher for 974573833245257412_7836001018745200024_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_7836001018745200024_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_6017148334314909027_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Added result fetcher for 974573833245257412_6017148334314909027_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_6017148334314909027_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5875873676127914865_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5875873676127914865_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5875873676127914865_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_5011235230273436863_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO ProgressReporter$: Added result fetcher for 974573833245257412_5011235230273436863_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:13 INFO DatabricksFileSystemV2Factory: Creating abfss file system for abfss://trusted@zedlsbi01t.dfs.core.windows.net 23/06/28 07:51:14 INFO AzureBlobFileSystem:V3: Initializing AzureBlobFileSystem for abfss://trusted@zedlsbi01t.dfs.core.windows.net/ with credential = ClientCredsTokenProvider;-298583247;574728210 with jvmId = 2036 23/06/28 07:51:14 INFO ClusterLoadMonitor: Added query with execution ID:163. Current active queries:1 23/06/28 07:51:14 INFO HiveMetaStore: 4: get_database: trusted 23/06/28 07:51:14 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:51:14 INFO HiveMetaStore: 4: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:14 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:14 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:14 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:14 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:14.521Z","run":{"runId":"9c900ecb-d78e-4b6f-9321-a4ce6699f5fe","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.CreateNamespace","num-children":1,"name":0,"ifNotExists":true,"properties":null},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedNamespace","num-children":0,"catalog":null,"namespace":"[trusted]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_create_namespace","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:14 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:14 INFO ClusterLoadMonitor: Removed query with execution ID:163. Current active queries:0 23/06/28 07:51:14 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:14 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:51:14.568Z","run":{"runId":"9c900ecb-d78e-4b6f-9321-a4ce6699f5fe","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.CreateNamespace","num-children":1,"name":0,"ifNotExists":true,"properties":null},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedNamespace","num-children":0,"catalog":null,"namespace":"[trusted]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_create_namespace","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:14 INFO HiveMetaStore: 4: get_table : db=trusted tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:51:14 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=trusted tbl=lms_lmsuu_mdl_groups_members 23/06/28 07:51:14 INFO DeltaLog: Loading version 60 starting from checkpoint version 60. 23/06/28 07:51:14 INFO SnapshotEdge: [tableId=41882988-6916-4192-9ef2-0764e84dd78c] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log, version=60, metadata=Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)), logSegment=LogSegment(dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log,60,WrappedArray(),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000060.checkpoint.parquet; isDirectory=false; length=29625; replication=1; blocksize=268435456; modification_time=1687937289000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A938E70CD4'}),Some(60),1687937287000), checksumOpt=Some(VersionChecksum(Some(003f2dbf-01da-4336-ad93-82707cb2039f),86107,6,None,None,1,1,Some(List()),Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@55d67ec4,[J@33c805e4)),None,Some(List(AddFile(part-00000-6b9b3a1d-a8da-45de-a4b3-8cd50f6ca176-c000.snappy.parquet,Map(),1392,1658882351000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-0321307b-64b5-4330-9ea7-f4a1546c9fc7-c000.snappy.parquet,Map(),1392,1658899177000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-78d99c18-4585-493e-80ff-883714b186ca-c000.snappy.parquet,Map(),1392,1642649707000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-47a3c79f-1f53-4b6a-80c4-bd71f2f757a8-c000.snappy.parquet,Map(),79147,1658899170000,false,{"numRecords":5752,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2022-01-20T14:30:28.955Z","_DLTrustedZoneTimeStamp":"2022-01-20T14:35:20.000Z","_RecordStart":"2022-01-20T14:35:20.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":0,"_RecordCurrent":1},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2022-07-26T18:00:15.455Z","_DLTrustedZoneTimeStamp":"2022-07-26T18:11:19.000Z","_RecordStart":"2022-07-26T18:10:50.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":1,"_RecordCurrent":1},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0,"_DLTrustedZoneTimeStamp":0,"_RecordStart":0,"_RecordEnd":0,"_RecordDeleted":0,"_RecordCurrent":0}},null,null), AddFile(part-00000-19db40fb-349f-41e5-af51-2ddf2517f348-c000.snappy.parquet,Map(),1392,1643247051000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-b3bce997-2d4c-43b0-94ba-039b842395a3-c000.snappy.parquet,Map(),1392,1642649740000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null)))))) 23/06/28 07:51:14 INFO ClusterLoadMonitor: Added query with execution ID:164. Current active queries:1 23/06/28 07:51:14 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:14 INFO CodeGenerator: Code generated in 18.509849 ms 23/06/28 07:51:14 INFO ClusterLoadMonitor: Removed query with execution ID:164. Current active queries:0 23/06/28 07:51:14 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:14 INFO HiveMetaStore: 5: get_database: trusted 23/06/28 07:51:14 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:51:14 INFO HiveMetaStore: 5: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:14 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:14 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:14 INFO HiveMetaStore: 5: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:14 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:14 INFO HiveMetaStore: 6: get_database: trusted 23/06/28 07:51:14 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:51:14 INFO HiveMetaStore: 6: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:14 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:15 INFO HiveClientImpl: Warehouse location for Hive client (version 0.13.1) is dbfs:/user/hive/warehouse 23/06/28 07:51:15 INFO HiveMetaStore: No user is added in admin role, since config is empty 23/06/28 07:51:15 INFO HiveMetaStore: 7: get_database: raw 23/06/28 07:51:15 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:51:15 INFO HiveMetaStore: 7: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:15 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:15 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:15 INFO HiveMetaStore: 6: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:15 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:15 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:15 INFO HiveMetaStore: 7: get_multi_table : db=raw tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:15 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=raw tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:15 INFO ClusterLoadMonitor: Added query with execution ID:165. Current active queries:1 23/06/28 07:51:15 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:15 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_70 stored as values in memory (estimated size 508.5 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_70_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:15 INFO SparkContext: Created broadcast 70 from broadcast at DeltaLog.scala:634 23/06/28 07:51:15 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 29625) 23/06/28 07:51:15 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:15.479Z","run":{"runId":"f200eb6a-d0e3-4882-8df0-efcba40cff35","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.MergeIntoCommandEdge","num-children":0,"source":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"STG","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"merge_key","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2896,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, raw]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649443000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":false,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"target":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"MN","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2915,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2916,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2917,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2918,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"targetFileIndex":null,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2896,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],"matchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoMatchedUpdateClause","num-children":13,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"actions":[1,2,3,4,5,6,7,8,9,10,11,12]},{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2917,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2918,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"notMatchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoNotMatchedInsertClause","num-children":12,"actions":[0,1,2,3,4,5,6,7,8,9,10,11]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"notMatchedBySourceClauses":[],"migratedSchema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"trackHighWaterMarks":{"object":"scala.collection.immutable.Set$EmptySet$"}}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_merge_into_command_edge","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:15 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:15 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:15 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:15 INFO CodeGenerator: Code generated in 41.298147 ms 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_71 stored as values in memory (estimated size 607.9 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:51:15 INFO SparkContext: Created broadcast 71 from $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604 23/06/28 07:51:15 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:15 INFO DAGScheduler: Registering RDD 239 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 27 23/06/28 07:51:15 INFO DAGScheduler: Got map stage job 44 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:51:15 INFO DAGScheduler: Final stage: ShuffleMapStage 118 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:51:15 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:15 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:15 INFO DAGScheduler: Submitting ShuffleMapStage 118 (MapPartitionsRDD[239] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:51:15 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:15 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:15 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:15 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:15 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 118 (MapPartitionsRDD[239] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:15 INFO TaskSchedulerImpl: Adding task set 118.0 with 1 tasks resource profile 0 23/06/28 07:51:15 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 974573833245257412, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 974573833245257412. Created 974573833245257412 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:51:15 INFO FairSchedulableBuilder: Added task set TaskSet_118.0 tasks to pool 974573833245257412 23/06/28 07:51:15 INFO TaskSetManager: Starting task 0.0 in stage 118.0 (TID 1284) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_72 stored as values in memory (estimated size 322.9 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO MemoryStore: Block broadcast_72_piece0 stored as bytes in memory (estimated size 78.6 KiB, free 750.6 GiB) 23/06/28 07:51:15 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on 10.60.24.15:40507 (size: 78.6 KiB, free: 750.7 GiB) 23/06/28 07:51:15 INFO SparkContext: Created broadcast 72 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:15 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on 10.60.24.10:41041 (size: 78.6 KiB, free: 487.7 GiB) 23/06/28 07:51:15 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:51:16 INFO TaskSetManager: Finished task 0.0 in stage 118.0 (TID 1284) in 579 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:16 INFO TaskSchedulerImpl: Removed TaskSet 118.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:16 INFO DAGScheduler: ShuffleMapStage 118 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.586 s 23/06/28 07:51:16 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:16 INFO DAGScheduler: running: Set() 23/06/28 07:51:16 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:16 INFO DAGScheduler: failed: Set() 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:16 INFO CodeGenerator: Code generated in 24.665755 ms 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:16 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":9591,"lifetime":374931},"10.60.24.13":{"diskUsage":0,"lifetime":375201}}, ioCacheNumScanTasks={"numLocalScanTasks":6,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:16 INFO CodeGenerator: Code generated in 27.199067 ms 23/06/28 07:51:16 INFO DAGScheduler: Registering RDD 249 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 28 23/06/28 07:51:16 INFO DAGScheduler: Got map stage job 45 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:51:16 INFO DAGScheduler: Final stage: ShuffleMapStage 120 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:51:16 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 119) 23/06/28 07:51:16 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:16 INFO DAGScheduler: Submitting ShuffleMapStage 120 (MapPartitionsRDD[249] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:16 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:16 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:16 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:16 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 120 (MapPartitionsRDD[249] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:16 INFO TaskSchedulerImpl: Adding task set 120.0 with 1 tasks resource profile 0 23/06/28 07:51:16 INFO FairSchedulableBuilder: Added task set TaskSet_120.0 tasks to pool 974573833245257412 23/06/28 07:51:16 INFO TaskSetManager: Starting task 0.0 in stage 120.0 (TID 1285) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_73 stored as values in memory (estimated size 468.3 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in memory (estimated size 118.9 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on 10.60.24.15:40507 (size: 118.9 KiB, free: 750.7 GiB) 23/06/28 07:51:16 INFO SparkContext: Created broadcast 73 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:16 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on 10.60.24.10:41041 (size: 118.9 KiB, free: 487.7 GiB) 23/06/28 07:51:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 27 to 10.60.24.10:44288 23/06/28 07:51:16 INFO BlockManagerInfo: Added rdd_246_0 in memory on 10.60.24.10:41041 (size: 1421.0 B, free: 487.7 GiB) 23/06/28 07:51:16 INFO TaskSetManager: Finished task 0.0 in stage 120.0 (TID 1285) in 72 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:16 INFO TaskSchedulerImpl: Removed TaskSet 120.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:16 INFO DAGScheduler: ShuffleMapStage 120 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.092 s 23/06/28 07:51:16 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:16 INFO DAGScheduler: running: Set() 23/06/28 07:51:16 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:16 INFO DAGScheduler: failed: Set() 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:16 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:51:16 INFO DAGScheduler: Got job 46 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:51:16 INFO DAGScheduler: Final stage: ResultStage 123 (first at Snapshot.scala:271) 23/06/28 07:51:16 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 122) 23/06/28 07:51:16 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:16 INFO DAGScheduler: Submitting ResultStage 123 (MapPartitionsRDD[251] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:16 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:16 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:16 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:16 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 123 (MapPartitionsRDD[251] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:16 INFO TaskSchedulerImpl: Adding task set 123.0 with 1 tasks resource profile 0 23/06/28 07:51:16 INFO FairSchedulableBuilder: Added task set TaskSet_123.0 tasks to pool 974573833245257412 23/06/28 07:51:16 INFO TaskSetManager: Starting task 0.0 in stage 123.0 (TID 1286) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_74 stored as values in memory (estimated size 414.5 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in memory (estimated size 107.6 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on 10.60.24.15:40507 (size: 107.6 KiB, free: 750.7 GiB) 23/06/28 07:51:16 INFO SparkContext: Created broadcast 74 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:16 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on 10.60.24.13:44035 (size: 107.6 KiB, free: 487.7 GiB) 23/06/28 07:51:16 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 28 to 10.60.24.13:58548 23/06/28 07:51:16 INFO TaskSetManager: Finished task 0.0 in stage 123.0 (TID 1286) in 64 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:16 INFO TaskSchedulerImpl: Removed TaskSet 123.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:16 INFO DAGScheduler: ResultStage 123 (first at Snapshot.scala:271) finished in 0.079 s 23/06/28 07:51:16 INFO DAGScheduler: Job 46 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:16 INFO TaskSchedulerImpl: Killing all running tasks in stage 123: Stage finished 23/06/28 07:51:16 INFO DAGScheduler: Job 46 finished: first at Snapshot.scala:271, took 0.080544 s 23/06/28 07:51:16 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:16 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_75 stored as values in memory (estimated size 508.5 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:16 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:16 INFO SparkContext: Created broadcast 75 from broadcast at BitmapFilter.scala:230 23/06/28 07:51:17 INFO SnapshotEdge: [tableId=32485164-a88a-4b48-ac48-c8898e21a3ee] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_76 stored as values in memory (estimated size 508.5 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_76_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 76 from broadcast at DeltaLog.scala:634 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 24.751286 ms 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 19.409845 ms 23/06/28 07:51:17 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:17 INFO DAGScheduler: Got job 47 (executeCollect at DatasetRefCache.scala:71) with 13 output partitions 23/06/28 07:51:17 INFO DAGScheduler: Final stage: ResultStage 124 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:51:17 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:17 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:17 INFO DAGScheduler: Submitting ResultStage 124 (MapPartitionsRDD[254] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:51:17 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:17 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:17 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:17 INFO DAGScheduler: Submitting 13 missing tasks from ResultStage 124 (MapPartitionsRDD[254] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Adding task set 124.0 with 13 tasks resource profile 0 23/06/28 07:51:17 INFO FairSchedulableBuilder: Added task set TaskSet_124.0 tasks to pool 974573833245257412 23/06/28 07:51:17 INFO TaskSetManager: Starting task 0.0 in stage 124.0 (TID 1287) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 1.0 in stage 124.0 (TID 1288) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 2.0 in stage 124.0 (TID 1289) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 3.0 in stage 124.0 (TID 1290) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 4.0 in stage 124.0 (TID 1291) (10.60.24.13, executor 1, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 5.0 in stage 124.0 (TID 1292) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 6.0 in stage 124.0 (TID 1293) (10.60.24.13, executor 1, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 7.0 in stage 124.0 (TID 1294) (10.60.24.10, executor 0, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 8.0 in stage 124.0 (TID 1295) (10.60.24.13, executor 1, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 9.0 in stage 124.0 (TID 1296) (10.60.24.10, executor 0, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 10.0 in stage 124.0 (TID 1297) (10.60.24.13, executor 1, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 11.0 in stage 124.0 (TID 1298) (10.60.24.10, executor 0, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 12.0 in stage 124.0 (TID 1299) (10.60.24.13, executor 1, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_77 stored as values in memory (estimated size 210.0 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in memory (estimated size 45.2 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on 10.60.24.15:40507 (size: 45.2 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 77 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on 10.60.24.10:41041 (size: 45.2 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on 10.60.24.13:44035 (size: 45.2 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 7.0 in stage 124.0 (TID 1294) in 26 ms on 10.60.24.10 (executor 0) (1/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 5.0 in stage 124.0 (TID 1292) in 26 ms on 10.60.24.10 (executor 0) (2/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 9.0 in stage 124.0 (TID 1296) in 27 ms on 10.60.24.10 (executor 0) (3/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 1.0 in stage 124.0 (TID 1288) in 28 ms on 10.60.24.10 (executor 0) (4/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 3.0 in stage 124.0 (TID 1290) in 30 ms on 10.60.24.10 (executor 0) (5/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 0.0 in stage 124.0 (TID 1287) in 31 ms on 10.60.24.13 (executor 1) (6/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 8.0 in stage 124.0 (TID 1295) in 31 ms on 10.60.24.13 (executor 1) (7/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 12.0 in stage 124.0 (TID 1299) in 31 ms on 10.60.24.13 (executor 1) (8/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 10.0 in stage 124.0 (TID 1297) in 31 ms on 10.60.24.13 (executor 1) (9/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 4.0 in stage 124.0 (TID 1291) in 31 ms on 10.60.24.13 (executor 1) (10/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 11.0 in stage 124.0 (TID 1298) in 31 ms on 10.60.24.10 (executor 0) (11/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 2.0 in stage 124.0 (TID 1289) in 32 ms on 10.60.24.13 (executor 1) (12/13) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 6.0 in stage 124.0 (TID 1293) in 33 ms on 10.60.24.13 (executor 1) (13/13) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Removed TaskSet 124.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:17 INFO DAGScheduler: ResultStage 124 (executeCollect at DatasetRefCache.scala:71) finished in 0.043 s 23/06/28 07:51:17 INFO DAGScheduler: Job 47 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:17 INFO TaskSchedulerImpl: Killing all running tasks in stage 124: Stage finished 23/06/28 07:51:17 INFO DAGScheduler: Job 47 finished: executeCollect at DatasetRefCache.scala:71, took 0.044750 s 23/06/28 07:51:17 INFO FileSourceStrategy: Pushed Filters: IsNotNull(add.dataChange) 23/06/28 07:51:17 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(add#3507.dataChange) 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 28.820856 ms 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_78 stored as values in memory (estimated size 594.9 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in memory (estimated size 16.3 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on 10.60.24.15:40507 (size: 16.3 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 78 from executeCollect at OptimizeLocalUnion.scala:71 23/06/28 07:51:17 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:17 INFO SparkContext: Starting job: executeCollect at OptimizeLocalUnion.scala:71 23/06/28 07:51:17 INFO DAGScheduler: Got job 48 (executeCollect at OptimizeLocalUnion.scala:71) with 1 output partitions 23/06/28 07:51:17 INFO DAGScheduler: Final stage: ResultStage 125 (executeCollect at OptimizeLocalUnion.scala:71) 23/06/28 07:51:17 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:17 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:17 INFO DAGScheduler: Submitting ResultStage 125 (UnionRDD[258] at executeCollect at OptimizeLocalUnion.scala:71), which has no missing parents 23/06/28 07:51:17 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:17 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:17 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:17 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 125 (UnionRDD[258] at executeCollect at OptimizeLocalUnion.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Adding task set 125.0 with 1 tasks resource profile 0 23/06/28 07:51:17 INFO FairSchedulableBuilder: Added task set TaskSet_125.0 tasks to pool 974573833245257412 23/06/28 07:51:17 INFO TaskSetManager: Starting task 0.0 in stage 125.0 (TID 1300) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_79 stored as values in memory (estimated size 254.0 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_79_piece0 stored as bytes in memory (estimated size 59.2 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on 10.60.24.15:40507 (size: 59.2 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 79 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on 10.60.24.10:41041 (size: 59.2 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on 10.60.24.10:41041 (size: 16.3 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 0.0 in stage 125.0 (TID 1300) in 45 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Removed TaskSet 125.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:17 INFO DAGScheduler: ResultStage 125 (executeCollect at OptimizeLocalUnion.scala:71) finished in 0.048 s 23/06/28 07:51:17 INFO DAGScheduler: Job 48 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:17 INFO TaskSchedulerImpl: Killing all running tasks in stage 125: Stage finished 23/06/28 07:51:17 INFO DAGScheduler: Job 48 finished: executeCollect at OptimizeLocalUnion.scala:71, took 0.049429 s 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 8.064967 ms 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 21.838154 ms 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_80 stored as values in memory (estimated size 8.9 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in memory (estimated size 4.1 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on 10.60.24.15:40507 (size: 4.1 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 80 from collect at LowShuffleMergeExecutor.scala:638 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 10.121622 ms 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_81 stored as values in memory (estimated size 6.9 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_81_piece0 stored as bytes in memory (estimated size 3.2 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on 10.60.24.15:40507 (size: 3.2 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 81 from collect at LowShuffleMergeExecutor.scala:638 23/06/28 07:51:17 INFO DataSourceStrategy: Pruning directories with: 23/06/28 07:51:17 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:17 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:17 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:17 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 8.496156 ms 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 9.892122 ms 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_82 stored as values in memory (estimated size 551.1 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_83 stored as values in memory (estimated size 544.5 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on 10.60.24.15:40507 (size: 15.3 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 82 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in memory (estimated size 15.1 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on 10.60.24.15:40507 (size: 15.1 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 83 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:17 INFO SnapshotEdge: [tableId=5d0ae5b6-42a3-47d1-8d94-be2429376ace] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:51:17 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:17 INFO DAGScheduler: Registering RDD 262 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 29 23/06/28 07:51:17 INFO DAGScheduler: Got map stage job 49 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:17 INFO DAGScheduler: Final stage: ShuffleMapStage 126 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:17 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:17 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:17 INFO DAGScheduler: Submitting ShuffleMapStage 126 (MapPartitionsRDD[262] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:17 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:17 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:17 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:17 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 126 (MapPartitionsRDD[262] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Adding task set 126.0 with 1 tasks resource profile 0 23/06/28 07:51:17 INFO FairSchedulableBuilder: Added task set TaskSet_126.0 tasks to pool 974573833245257412 23/06/28 07:51:17 INFO TaskSetManager: Starting task 0.0 in stage 126.0 (TID 1301) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_84 stored as values in memory (estimated size 183.6 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO FileSourceStrategy: Pushed Filters: IsNotNull(add.dataChange) 23/06/28 07:51:17 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(add#3750.dataChange) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in memory (estimated size 45.0 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on 10.60.24.15:40507 (size: 45.0 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 84 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on 10.60.24.13:44035 (size: 45.0 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 30.718827 ms 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_85 stored as values in memory (estimated size 598.5 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on 10.60.24.15:40507 (size: 16.4 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 85 from executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:17 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:17 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:17 INFO DAGScheduler: Got job 50 (executeCollect at DatasetRefCache.scala:71) with 1 output partitions 23/06/28 07:51:17 INFO DAGScheduler: Final stage: ResultStage 127 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:51:17 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:17 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:17 INFO DAGScheduler: Submitting ResultStage 127 (MapPartitionsRDD[265] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:17 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:17 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:17 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:17 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 127 (MapPartitionsRDD[265] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Adding task set 127.0 with 1 tasks resource profile 0 23/06/28 07:51:17 INFO FairSchedulableBuilder: Added task set TaskSet_127.0 tasks to pool 974573833245257412 23/06/28 07:51:17 INFO TaskSetManager: Starting task 0.0 in stage 127.0 (TID 1302) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_86 stored as values in memory (estimated size 260.8 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in memory (estimated size 60.2 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on 10.60.24.15:40507 (size: 60.2 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 86 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on 10.60.24.10:41041 (size: 60.2 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on 10.60.24.10:41041 (size: 16.4 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on 10.60.24.13:44035 (size: 15.1 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 0.0 in stage 127.0 (TID 1302) in 179 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Removed TaskSet 127.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:17 INFO DAGScheduler: ResultStage 127 (executeCollect at DatasetRefCache.scala:71) finished in 0.183 s 23/06/28 07:51:17 INFO DAGScheduler: Job 50 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:17 INFO TaskSchedulerImpl: Killing all running tasks in stage 127: Stage finished 23/06/28 07:51:17 INFO DAGScheduler: Job 50 finished: executeCollect at DatasetRefCache.scala:71, took 0.183721 s 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:17 INFO CodeGenerator: Code generated in 8.381832 ms 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:17 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:17 INFO DAGScheduler: Registering RDD 269 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 30 23/06/28 07:51:17 INFO DAGScheduler: Got map stage job 51 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:17 INFO DAGScheduler: Final stage: ShuffleMapStage 128 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:17 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:17 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:17 INFO DAGScheduler: Submitting ShuffleMapStage 128 (MapPartitionsRDD[269] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:17 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:17 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:17 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:17 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:17 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 128 (MapPartitionsRDD[269] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:17 INFO TaskSchedulerImpl: Adding task set 128.0 with 6 tasks resource profile 0 23/06/28 07:51:17 INFO FairSchedulableBuilder: Added task set TaskSet_128.0 tasks to pool 974573833245257412 23/06/28 07:51:17 INFO TaskSetManager: Starting task 0.0 in stage 128.0 (TID 1303) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 4.0 in stage 128.0 (TID 1304) (10.60.24.10, executor 0, partition 4, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 1.0 in stage 128.0 (TID 1305) (10.60.24.13, executor 1, partition 1, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 5.0 in stage 128.0 (TID 1306) (10.60.24.10, executor 0, partition 5, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 2.0 in stage 128.0 (TID 1307) (10.60.24.13, executor 1, partition 2, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO TaskSetManager: Starting task 3.0 in stage 128.0 (TID 1308) (10.60.24.13, executor 1, partition 3, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_87 stored as values in memory (estimated size 197.3 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO MemoryStore: Block broadcast_87_piece0 stored as bytes in memory (estimated size 50.5 KiB, free 750.6 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on 10.60.24.15:40507 (size: 50.5 KiB, free: 750.7 GiB) 23/06/28 07:51:17 INFO SparkContext: Created broadcast 87 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on 10.60.24.10:41041 (size: 50.5 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on 10.60.24.13:44035 (size: 50.5 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on 10.60.24.10:41041 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on 10.60.24.13:44035 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 5.0 in stage 128.0 (TID 1306) in 155 ms on 10.60.24.10 (executor 0) (1/6) 23/06/28 07:51:17 INFO TaskSetManager: Finished task 4.0 in stage 128.0 (TID 1304) in 182 ms on 10.60.24.10 (executor 0) (2/6) 23/06/28 07:51:19 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:51:21 INFO TaskSetManager: Finished task 0.0 in stage 128.0 (TID 1303) in 3224 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 2.0 in stage 128.0 (TID 1307) in 3223 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 3.0 in stage 128.0 (TID 1308) in 3223 ms on 10.60.24.13 (executor 1) (5/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 1.0 in stage 128.0 (TID 1305) in 3225 ms on 10.60.24.13 (executor 1) (6/6) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Removed TaskSet 128.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:21 INFO DAGScheduler: ShuffleMapStage 128 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 3.229 s 23/06/28 07:51:21 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:21 INFO DAGScheduler: running: Set(ShuffleMapStage 126) 23/06/28 07:51:21 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:21 INFO DAGScheduler: failed: Set() 23/06/28 07:51:21 INFO ShufflePartitionsUtil: For shuffle(30), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:21 INFO DAGScheduler: Registering RDD 271 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 31 23/06/28 07:51:21 INFO DAGScheduler: Got map stage job 52 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:21 INFO DAGScheduler: Final stage: ShuffleMapStage 130 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:21 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 129) 23/06/28 07:51:21 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:21 INFO DAGScheduler: Submitting ShuffleMapStage 130 (MapPartitionsRDD[271] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:21 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:21 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:21 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:21 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:21 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 130 (MapPartitionsRDD[271] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Adding task set 130.0 with 6 tasks resource profile 0 23/06/28 07:51:21 INFO FairSchedulableBuilder: Added task set TaskSet_130.0 tasks to pool 974573833245257412 23/06/28 07:51:21 INFO TaskSetManager: Starting task 4.0 in stage 130.0 (TID 1309) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO TaskSetManager: Starting task 0.0 in stage 130.0 (TID 1310) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO TaskSetManager: Starting task 5.0 in stage 130.0 (TID 1311) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO TaskSetManager: Starting task 1.0 in stage 130.0 (TID 1312) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO TaskSetManager: Starting task 2.0 in stage 130.0 (TID 1313) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO TaskSetManager: Starting task 3.0 in stage 130.0 (TID 1314) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_88 stored as values in memory (estimated size 129.0 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in memory (estimated size 21.2 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on 10.60.24.15:40507 (size: 21.2 KiB, free: 750.7 GiB) 23/06/28 07:51:21 INFO SparkContext: Created broadcast 88 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on 10.60.24.13:44035 (size: 21.2 KiB, free: 487.7 GiB) 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on 10.60.24.10:41041 (size: 21.2 KiB, free: 487.7 GiB) 23/06/28 07:51:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 30 to 10.60.24.13:58548 23/06/28 07:51:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 30 to 10.60.24.10:44288 23/06/28 07:51:21 INFO TaskSetManager: Finished task 5.0 in stage 130.0 (TID 1311) in 17 ms on 10.60.24.10 (executor 0) (1/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 2.0 in stage 130.0 (TID 1313) in 17 ms on 10.60.24.13 (executor 1) (2/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 0.0 in stage 130.0 (TID 1310) in 17 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 1.0 in stage 130.0 (TID 1312) in 17 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 3.0 in stage 130.0 (TID 1314) in 18 ms on 10.60.24.13 (executor 1) (5/6) 23/06/28 07:51:21 INFO TaskSetManager: Finished task 4.0 in stage 130.0 (TID 1309) in 22 ms on 10.60.24.10 (executor 0) (6/6) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Removed TaskSet 130.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:21 INFO DAGScheduler: ShuffleMapStage 130 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.024 s 23/06/28 07:51:21 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:21 INFO DAGScheduler: running: Set(ShuffleMapStage 126) 23/06/28 07:51:21 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:21 INFO DAGScheduler: failed: Set() 23/06/28 07:51:21 INFO TaskSetManager: Finished task 0.0 in stage 126.0 (TID 1301) in 3605 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Removed TaskSet 126.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:21 INFO DAGScheduler: ShuffleMapStage 126 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 3.610 s 23/06/28 07:51:21 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:21 INFO DAGScheduler: running: Set() 23/06/28 07:51:21 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:21 INFO DAGScheduler: failed: Set() 23/06/28 07:51:21 INFO ShufflePartitionsUtil: For shuffle(29), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:21 INFO CodeGenerator: Code generated in 12.219102 ms 23/06/28 07:51:21 INFO DAGScheduler: Registering RDD 280 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 32 23/06/28 07:51:21 INFO DAGScheduler: Got map stage job 53 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:21 INFO DAGScheduler: Final stage: ShuffleMapStage 134 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:21 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 132, ShuffleMapStage 133) 23/06/28 07:51:21 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:21 INFO DAGScheduler: Submitting ShuffleMapStage 134 (MapPartitionsRDD[280] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:21 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:21 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:21 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:21 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:21 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 134 (MapPartitionsRDD[280] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Adding task set 134.0 with 1 tasks resource profile 0 23/06/28 07:51:21 INFO FairSchedulableBuilder: Added task set TaskSet_134.0 tasks to pool 974573833245257412 23/06/28 07:51:21 INFO TaskSetManager: Starting task 0.0 in stage 134.0 (TID 1315) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_89 stored as values in memory (estimated size 250.4 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_89_piece0 stored as bytes in memory (estimated size 75.0 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on 10.60.24.15:40507 (size: 75.0 KiB, free: 750.7 GiB) 23/06/28 07:51:21 INFO SparkContext: Created broadcast 89 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on 10.60.24.13:44035 (size: 75.0 KiB, free: 487.7 GiB) 23/06/28 07:51:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 29 to 10.60.24.13:58548 23/06/28 07:51:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 31 to 10.60.24.13:58548 23/06/28 07:51:21 INFO TaskSetManager: Finished task 0.0 in stage 134.0 (TID 1315) in 401 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Removed TaskSet 134.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:21 INFO DAGScheduler: ShuffleMapStage 134 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.438 s 23/06/28 07:51:21 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:21 INFO DAGScheduler: running: Set() 23/06/28 07:51:21 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:21 INFO DAGScheduler: failed: Set() 23/06/28 07:51:21 INFO ShufflePartitionsUtil: For shuffle(32), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:21 INFO DAGScheduler: Registering RDD 284 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 33 23/06/28 07:51:21 INFO DAGScheduler: Got map stage job 54 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:21 INFO DAGScheduler: Final stage: ShuffleMapStage 139 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:21 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 138) 23/06/28 07:51:21 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:21 INFO DAGScheduler: Submitting ShuffleMapStage 139 (MapPartitionsRDD[284] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:21 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:21 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:21 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:21 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:21 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 139 (MapPartitionsRDD[284] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:21 INFO TaskSchedulerImpl: Adding task set 139.0 with 1 tasks resource profile 0 23/06/28 07:51:21 INFO FairSchedulableBuilder: Added task set TaskSet_139.0 tasks to pool 974573833245257412 23/06/28 07:51:21 INFO TaskSetManager: Starting task 0.0 in stage 139.0 (TID 1316) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_90 stored as values in memory (estimated size 245.1 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in memory (estimated size 72.0 KiB, free 750.6 GiB) 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on 10.60.24.15:40507 (size: 72.0 KiB, free: 750.7 GiB) 23/06/28 07:51:21 INFO SparkContext: Created broadcast 90 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:21 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on 10.60.24.10:41041 (size: 72.0 KiB, free: 487.7 GiB) 23/06/28 07:51:21 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 32 to 10.60.24.10:44288 23/06/28 07:51:22 INFO TaskSetManager: Finished task 0.0 in stage 139.0 (TID 1316) in 427 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:22 INFO TaskSchedulerImpl: Removed TaskSet 139.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:22 INFO DAGScheduler: ShuffleMapStage 139 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.437 s 23/06/28 07:51:22 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:22 INFO DAGScheduler: running: Set() 23/06/28 07:51:22 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:22 INFO DAGScheduler: failed: Set() 23/06/28 07:51:22 INFO ShufflePartitionsUtil: For shuffle(33), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:22 INFO CodeGenerator: Code generated in 5.62764 ms 23/06/28 07:51:22 INFO SparkContext: Starting job: collect at LowShuffleMergeExecutor.scala:638 23/06/28 07:51:22 INFO DAGScheduler: Got job 55 (collect at LowShuffleMergeExecutor.scala:638) with 1 output partitions 23/06/28 07:51:22 INFO DAGScheduler: Final stage: ResultStage 145 (collect at LowShuffleMergeExecutor.scala:638) 23/06/28 07:51:22 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 144) 23/06/28 07:51:22 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:22 INFO DAGScheduler: Submitting ResultStage 145 (MapPartitionsRDD[290] at collect at LowShuffleMergeExecutor.scala:638), which has no missing parents 23/06/28 07:51:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:22 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:22 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:22 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:22 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 145 (MapPartitionsRDD[290] at collect at LowShuffleMergeExecutor.scala:638) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:22 INFO TaskSchedulerImpl: Adding task set 145.0 with 1 tasks resource profile 0 23/06/28 07:51:22 INFO FairSchedulableBuilder: Added task set TaskSet_145.0 tasks to pool 974573833245257412 23/06/28 07:51:22 INFO TaskSetManager: Starting task 0.0 in stage 145.0 (TID 1317) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:22 INFO MemoryStore: Block broadcast_91 stored as values in memory (estimated size 278.8 KiB, free 750.6 GiB) 23/06/28 07:51:22 INFO MemoryStore: Block broadcast_91_piece0 stored as bytes in memory (estimated size 82.0 KiB, free 750.6 GiB) 23/06/28 07:51:22 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on 10.60.24.15:40507 (size: 82.0 KiB, free: 750.7 GiB) 23/06/28 07:51:22 INFO SparkContext: Created broadcast 91 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:22 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on 10.60.24.13:44035 (size: 82.0 KiB, free: 487.7 GiB) 23/06/28 07:51:22 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 33 to 10.60.24.13:58548 23/06/28 07:51:22 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on 10.60.24.13:44035 (size: 3.2 KiB, free: 487.7 GiB) 23/06/28 07:51:22 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on 10.60.24.13:44035 (size: 13.7 KiB, free: 487.7 GiB) 23/06/28 07:51:22 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on 10.60.24.13:44035 (size: 4.1 KiB, free: 487.7 GiB) 23/06/28 07:51:22 INFO TaskSetManager: Finished task 0.0 in stage 145.0 (TID 1317) in 285 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:22 INFO TaskSchedulerImpl: Removed TaskSet 145.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:22 INFO DAGScheduler: ResultStage 145 (collect at LowShuffleMergeExecutor.scala:638) finished in 0.308 s 23/06/28 07:51:22 INFO DAGScheduler: Job 55 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:22 INFO TaskSchedulerImpl: Killing all running tasks in stage 145: Stage finished 23/06/28 07:51:22 INFO DAGScheduler: Job 55 finished: collect at LowShuffleMergeExecutor.scala:638, took 0.309365 s 23/06/28 07:51:22 INFO CodeGenerator: Code generated in 15.230748 ms 23/06/28 07:51:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:22 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_31_1628468067) 23/06/28 07:51:22 WARN BlockManager: Asked to remove block spark_join_relation_31_1628468067, which does not exist 23/06/28 07:51:22 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":87844,"lifetime":380964},"10.60.24.13":{"diskUsage":74797,"lifetime":381234}}, ioCacheNumScanTasks={"numLocalScanTasks":15,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:22 INFO FileSizeAutoTuner: Rewrite-heavy auto-detection result in 295 ms: {"isRewriteHeavy":true,"numRewriteOpsFound":10,"numDataChangeOpsFound":10,"numNonDataChangeOpsFound":0,"autoDetectNumHistoryOps":10,"autoDetectRewriteOpsFraction":0.9} 23/06/28 07:51:22 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"rewrites","tunedConfs":{"spark.databricks.delta.optimize.maxFileSize":"67108864","spark.databricks.delta.optimize.minFileSize":"33554432","spark.databricks.delta.autoCompact.minFileSize":"8388608","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimizeWrite.binSize":"192"}} 23/06/28 07:51:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:51:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:22 INFO DataSourceStrategy: Pruning directories with: 23/06/28 07:51:22 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:22 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:22 INFO FileSourceStrategy: Pushed Filters: IsNotNull(_databricks_internal_edge_computed_column_skip_row),Not(EqualTo(_databricks_internal_edge_computed_column_skip_row,true)) 23/06/28 07:51:22 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(_databricks_internal_edge_computed_column_skip_row#3878),NOT _databricks_internal_edge_computed_column_skip_row#3878 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:23 INFO CodeGenerator: Code generated in 9.85625 ms 23/06/28 07:51:23 INFO CodeGenerator: Code generated in 9.265687 ms 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_92 stored as values in memory (estimated size 546.9 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_93 stored as values in memory (estimated size 560.9 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on 10.60.24.15:40507 (size: 15.3 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 92 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in memory (estimated size 15.5 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on 10.60.24.15:40507 (size: 15.5 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 93 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:23 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:23 WARN DeltaParquetFileFormat: Forcing use of DatabricksVectorizedParquetRecordReader and disabling native reader because special columns are requested in schema: StructField(component,StringType,true)StructField(groupid,LongType,true)StructField(id,LongType,true)StructField(itemid,LongType,true)StructField(timeadded,LongType,true)StructField(userid,LongType,true)StructField(_DLRawZoneTimeStamp,TimestampType,true)StructField(_DLTrustedZoneTimeStamp,TimestampType,true)StructField(_RecordStart,TimestampType,true)StructField(_RecordEnd,TimestampType,true)StructField(_RecordDeleted,IntegerType,true)StructField(_RecordCurrent,IntegerType,true)StructField(_databricks_internal_edge_computed_column_skip_row,BooleanType,true) 23/06/28 07:51:23 INFO DAGScheduler: Registering RDD 294 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 34 23/06/28 07:51:23 INFO DAGScheduler: Got map stage job 56 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:51:23 INFO DAGScheduler: Final stage: ShuffleMapStage 146 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:51:23 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:23 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:23 INFO DAGScheduler: Submitting ShuffleMapStage 146 (MapPartitionsRDD[294] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:23 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:23 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:23 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:23 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 146 (MapPartitionsRDD[294] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Adding task set 146.0 with 1 tasks resource profile 0 23/06/28 07:51:23 INFO FairSchedulableBuilder: Added task set TaskSet_146.0 tasks to pool 974573833245257412 23/06/28 07:51:23 INFO TaskSetManager: Starting task 0.0 in stage 146.0 (TID 1318) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_94 stored as values in memory (estimated size 184.6 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in memory (estimated size 45.4 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on 10.60.24.15:40507 (size: 45.4 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 94 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on 10.60.24.13:44035 (size: 45.4 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on 10.60.24.13:44035 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:23 INFO CodeGenerator: Code generated in 8.823996 ms 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:23 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:23 INFO DAGScheduler: Registering RDD 298 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 35 23/06/28 07:51:23 INFO DAGScheduler: Got map stage job 57 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:51:23 INFO DAGScheduler: Final stage: ShuffleMapStage 147 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:51:23 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:23 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:23 INFO DAGScheduler: Submitting ShuffleMapStage 147 (MapPartitionsRDD[298] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:23 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:23 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:23 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:23 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 147 (MapPartitionsRDD[298] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Adding task set 147.0 with 1 tasks resource profile 0 23/06/28 07:51:23 INFO FairSchedulableBuilder: Added task set TaskSet_147.0 tasks to pool 974573833245257412 23/06/28 07:51:23 INFO TaskSetManager: Starting task 0.0 in stage 147.0 (TID 1319) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_95 stored as values in memory (estimated size 200.0 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_95_piece0 stored as bytes in memory (estimated size 51.2 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on 10.60.24.15:40507 (size: 51.2 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 95 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on 10.60.24.10:41041 (size: 51.2 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on 10.60.24.10:41041 (size: 15.5 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO TaskSetManager: Finished task 0.0 in stage 146.0 (TID 1318) in 115 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Removed TaskSet 146.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:23 INFO DAGScheduler: ShuffleMapStage 146 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.118 s 23/06/28 07:51:23 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:23 INFO DAGScheduler: running: Set(ShuffleMapStage 147) 23/06/28 07:51:23 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:23 INFO DAGScheduler: failed: Set() 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:23 INFO TaskSetManager: Finished task 0.0 in stage 147.0 (TID 1319) in 244 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Removed TaskSet 147.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:23 INFO DAGScheduler: ShuffleMapStage 147 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.249 s 23/06/28 07:51:23 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:23 INFO DAGScheduler: running: Set() 23/06/28 07:51:23 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:23 INFO DAGScheduler: failed: Set() 23/06/28 07:51:23 INFO ShufflePartitionsUtil: For shuffle(35), advisory target size: 201326592, actual target size 50331648, minimum partition size: 50331648 23/06/28 07:51:23 INFO DAGScheduler: Registering RDD 300 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 36 23/06/28 07:51:23 INFO DAGScheduler: Got map stage job 58 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:51:23 INFO DAGScheduler: Final stage: ShuffleMapStage 149 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:51:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 148) 23/06/28 07:51:23 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:23 INFO DAGScheduler: Submitting ShuffleMapStage 149 (MapPartitionsRDD[300] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:23 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:23 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:23 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:23 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 149 (MapPartitionsRDD[300] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Adding task set 149.0 with 1 tasks resource profile 0 23/06/28 07:51:23 INFO FairSchedulableBuilder: Added task set TaskSet_149.0 tasks to pool 974573833245257412 23/06/28 07:51:23 INFO TaskSetManager: Starting task 0.0 in stage 149.0 (TID 1320) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_96 stored as values in memory (estimated size 129.3 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in memory (estimated size 21.5 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on 10.60.24.15:40507 (size: 21.5 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 96 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on 10.60.24.10:41041 (size: 21.5 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 35 to 10.60.24.10:44288 23/06/28 07:51:23 INFO TaskSetManager: Finished task 0.0 in stage 149.0 (TID 1320) in 20 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Removed TaskSet 149.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:23 INFO DAGScheduler: ShuffleMapStage 149 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.022 s 23/06/28 07:51:23 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:23 INFO DAGScheduler: running: Set() 23/06/28 07:51:23 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:23 INFO DAGScheduler: failed: Set() 23/06/28 07:51:23 INFO ShufflePartitionsUtil: For shuffle(34), advisory target size: 201326592, actual target size 50331648, minimum partition size: 50331648 23/06/28 07:51:23 INFO ParquetUtils: Using user defined output committer for Parquet: org.apache.spark.sql.parquet.DirectParquetOutputCommitter 23/06/28 07:51:23 INFO CodeGenerator: Code generated in 15.678487 ms 23/06/28 07:51:23 INFO SparkContext: Starting job: write at WriteIntoDeltaCommand.scala:70 23/06/28 07:51:23 INFO DAGScheduler: Got job 59 (write at WriteIntoDeltaCommand.scala:70) with 1 output partitions 23/06/28 07:51:23 INFO DAGScheduler: Final stage: ResultStage 153 (write at WriteIntoDeltaCommand.scala:70) 23/06/28 07:51:23 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 150, ShuffleMapStage 152) 23/06/28 07:51:23 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:23 INFO DAGScheduler: Submitting ResultStage 153 (MapPartitionsRDD[307] at write at WriteIntoDeltaCommand.scala:70), which has no missing parents 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:23 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:23 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:23 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:23 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 153 (MapPartitionsRDD[307] at write at WriteIntoDeltaCommand.scala:70) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Adding task set 153.0 with 1 tasks resource profile 0 23/06/28 07:51:23 INFO FairSchedulableBuilder: Added task set TaskSet_153.0 tasks to pool 974573833245257412 23/06/28 07:51:23 INFO TaskSetManager: Starting task 0.0 in stage 153.0 (TID 1321) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_97 stored as values in memory (estimated size 373.1 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in memory (estimated size 124.1 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on 10.60.24.15:40507 (size: 124.1 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 97 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on 10.60.24.13:44035 (size: 124.1 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 34 to 10.60.24.13:58548 23/06/28 07:51:23 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 36 to 10.60.24.13:58548 23/06/28 07:51:23 INFO TaskSetManager: Finished task 0.0 in stage 153.0 (TID 1321) in 299 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Removed TaskSet 153.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:23 INFO DAGScheduler: ResultStage 153 (write at WriteIntoDeltaCommand.scala:70) finished in 0.321 s 23/06/28 07:51:23 INFO DAGScheduler: Job 59 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:23 INFO TaskSchedulerImpl: Killing all running tasks in stage 153: Stage finished 23/06/28 07:51:23 INFO DAGScheduler: Job 59 finished: write at WriteIntoDeltaCommand.scala:70, took 0.322265 s 23/06/28 07:51:23 INFO FileFormatWriter: Start to commit write Job 8308edf3-fe4c-4edd-8446-913f9402f491. 23/06/28 07:51:23 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:51:23 INFO FileFormatWriter: Write Job 8308edf3-fe4c-4edd-8446-913f9402f491 committed. Elapsed time: 1 ms. 23/06/28 07:51:23 INFO FileFormatWriter: Finished processing stats for write job 8308edf3-fe4c-4edd-8446-913f9402f491. 23/06/28 07:51:23 INFO SparkContext: Starting job: collect at TransactionalWriteEdge.scala:576 23/06/28 07:51:23 INFO DAGScheduler: Got job 60 (collect at TransactionalWriteEdge.scala:576) with 1 output partitions 23/06/28 07:51:23 INFO DAGScheduler: Final stage: ResultStage 154 (collect at TransactionalWriteEdge.scala:576) 23/06/28 07:51:23 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:23 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:23 INFO DAGScheduler: Submitting ResultStage 154 (SQLExecutionRDD[309] at toRdd at TransactionalWriteEdge.scala:576), which has no missing parents 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:23 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:23 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:23 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:23 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 154 (SQLExecutionRDD[309] at toRdd at TransactionalWriteEdge.scala:576) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Adding task set 154.0 with 1 tasks resource profile 0 23/06/28 07:51:23 INFO FairSchedulableBuilder: Added task set TaskSet_154.0 tasks to pool 974573833245257412 23/06/28 07:51:23 INFO TaskSetManager: Starting task 0.0 in stage 154.0 (TID 1322) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_98 stored as values in memory (estimated size 148.9 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in memory (estimated size 30.2 KiB, free 750.6 GiB) 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on 10.60.24.15:40507 (size: 30.2 KiB, free: 750.7 GiB) 23/06/28 07:51:23 INFO SparkContext: Created broadcast 98 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:23 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on 10.60.24.13:44035 (size: 30.2 KiB, free: 487.7 GiB) 23/06/28 07:51:23 INFO TaskSetManager: Finished task 0.0 in stage 154.0 (TID 1322) in 16 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:23 INFO TaskSchedulerImpl: Removed TaskSet 154.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:23 INFO DAGScheduler: ResultStage 154 (collect at TransactionalWriteEdge.scala:576) finished in 0.018 s 23/06/28 07:51:23 INFO DAGScheduler: Job 60 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:23 INFO TaskSchedulerImpl: Killing all running tasks in stage 154: Stage finished 23/06/28 07:51:23 INFO DAGScheduler: Job 60 finished: collect at TransactionalWriteEdge.scala:576, took 0.018713 s 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:23 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_36_1628468067) 23/06/28 07:51:23 WARN BlockManager: Asked to remove block spark_join_relation_36_1628468067, which does not exist 23/06/28 07:51:23 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":89197,"lifetime":382318},"10.60.24.13":{"diskUsage":74852,"lifetime":382588}}, ioCacheNumScanTasks={"numLocalScanTasks":17,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:23 INFO SQLAppStatusListener: Recording DV cache-related metrics in usage logs: {"10.60.24.10":{"memCacheUsage":0,"diskCacheUsage":112,"lifetimeMs":382318},"10.60.24.13":{"memCacheUsage":0,"diskCacheUsage":0,"lifetimeMs":382588}} 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:23 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:23 INFO DataSourceStrategy: Pruning directories with: 23/06/28 07:51:23 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:23 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:23 INFO FileSourceStrategy: Pushed Filters: IsNotNull(_databricks_internal_edge_computed_column_skip_row),Not(EqualTo(_databricks_internal_edge_computed_column_skip_row,true)) 23/06/28 07:51:23 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(_databricks_internal_edge_computed_column_skip_row#3878),NOT _databricks_internal_edge_computed_column_skip_row#3878 23/06/28 07:51:23 INFO AzureBlobFileSystem:V3: FS_OP_DELETE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/deletion_vector_951f8cc5-5570-446d-9c87-2a1174b4d071.bin] Starting delete - recursive false 23/06/28 07:51:23 INFO AzureBlobFileSystem:V3: FS_OP_DELETE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/deletion_vector_951f8cc5-5570-446d-9c87-2a1174b4d071.bin] Delete successful. 23/06/28 07:51:23 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=b6dd64e0] Attempting to commit version 61 with 1 actions with WriteSerializable isolation level 23/06/28 07:51:23 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:51:24 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json using temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp 23/06/28 07:51:24 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp] Closing stream; size: 0 23/06/28 07:51:24 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp] Upload complete; size: 1943 23/06/28 07:51:24 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json] Starting rename. Issuing rename operation. 23/06/28 07:51:24 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json] Rename successful. 23/06/28 07:51:24 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.json.138c180f-95ad-43fc-aab2-eb630edad061.tmp to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json 23/06/28 07:51:24 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=b6dd64e0] Incremental commit: starting with snapshot version 60 23/06/28 07:51:24 INFO DeltaLog: Loading version 61 starting from checkpoint version 60. 23/06/28 07:51:24 INFO SnapshotEdge: [tableId=5d0ae5b6-42a3-47d1-8d94-be2429376ace] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log, version=61, metadata=Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)), logSegment=LogSegment(dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log,61,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json; isDirectory=false; length=1943; replication=1; blocksize=268435456; modification_time=1687938684000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC785C04B5"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000060.checkpoint.parquet; isDirectory=false; length=29625; replication=1; blocksize=268435456; modification_time=1687937289000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A938E70CD4'}),Some(60),1687938684000), checksumOpt=Some(VersionChecksum(Some(b6dd64e0-f25a-4992-9308-e68b3ea1b093),86107,6,None,None,1,1,Some(Stream()),Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@49d14876,[J@27dbabc)),None,Some(Stream(AddFile(part-00000-6b9b3a1d-a8da-45de-a4b3-8cd50f6ca176-c000.snappy.parquet,Map(),1392,1658882351000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-78d99c18-4585-493e-80ff-883714b186ca-c000.snappy.parquet,Map(),1392,1642649707000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-0321307b-64b5-4330-9ea7-f4a1546c9fc7-c000.snappy.parquet,Map(),1392,1658899177000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-47a3c79f-1f53-4b6a-80c4-bd71f2f757a8-c000.snappy.parquet,Map(),79147,1658899170000,false,{"numRecords":5752,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2022-01-20T14:30:28.955Z","_DLTrustedZoneTimeStamp":"2022-01-20T14:35:20.000Z","_RecordStart":"2022-01-20T14:35:20.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":0,"_RecordCurrent":1},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2022-07-26T18:00:15.455Z","_DLTrustedZoneTimeStamp":"2022-07-26T18:11:19.000Z","_RecordStart":"2022-07-26T18:10:50.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":1,"_RecordCurrent":1},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0,"_DLTrustedZoneTimeStamp":0,"_RecordStart":0,"_RecordEnd":0,"_RecordDeleted":0,"_RecordCurrent":0}},null,null), AddFile(part-00000-19db40fb-349f-41e5-af51-2ddf2517f348-c000.snappy.parquet,Map(),1392,1643247051000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-b3bce997-2d4c-43b0-94ba-039b842395a3-c000.snappy.parquet,Map(),1392,1642649740000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null)))))) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_99 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 99 from broadcast at DeltaLog.scala:634 23/06/28 07:51:24 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 29625) 23/06/28 07:51:24 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 1, totalFileSize: 1943) 23/06/28 07:51:24 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:24 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:24 INFO CodeGenerator: Code generated in 35.7671 ms 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_100 stored as values in memory (estimated size 610.3 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_100_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 100 from $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173 23/06/28 07:51:24 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:24 INFO CodeGenerator: Code generated in 19.178489 ms 23/06/28 07:51:24 INFO DAGScheduler: Registering RDD 316 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 37 23/06/28 07:51:24 INFO DAGScheduler: Got map stage job 61 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 2 output partitions 23/06/28 07:51:24 INFO DAGScheduler: Final stage: ShuffleMapStage 155 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:24 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:24 INFO DAGScheduler: Submitting ShuffleMapStage 155 (MapPartitionsRDD[316] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:24 INFO DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 155 (MapPartitionsRDD[316] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1)) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Adding task set 155.0 with 2 tasks resource profile 0 23/06/28 07:51:24 INFO FairSchedulableBuilder: Added task set TaskSet_155.0 tasks to pool 974573833245257412 23/06/28 07:51:24 INFO TaskSetManager: Starting task 0.0 in stage 155.0 (TID 1323) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:24 INFO TaskSetManager: Starting task 1.0 in stage 155.0 (TID 1324) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_101 stored as values in memory (estimated size 324.4 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_101_piece0 stored as bytes in memory (estimated size 79.4 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on 10.60.24.15:40507 (size: 79.4 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 101 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on 10.60.24.13:44035 (size: 79.4 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on 10.60.24.10:41041 (size: 79.4 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO TaskSetManager: Finished task 1.0 in stage 155.0 (TID 1324) in 21 ms on 10.60.24.13 (executor 1) (1/2) 23/06/28 07:51:24 INFO TaskSetManager: Finished task 0.0 in stage 155.0 (TID 1323) in 74 ms on 10.60.24.10 (executor 0) (2/2) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Removed TaskSet 155.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:24 INFO DAGScheduler: ShuffleMapStage 155 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.081 s 23/06/28 07:51:24 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:24 INFO DAGScheduler: running: Set() 23/06/28 07:51:24 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:24 INFO DAGScheduler: failed: Set() 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:24 INFO CodeGenerator: Code generated in 35.283719 ms 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:24 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":89197,"lifetime":383086},"10.60.24.13":{"diskUsage":74852,"lifetime":383356}}, ioCacheNumScanTasks={"numLocalScanTasks":18,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:24 INFO CodeGenerator: Code generated in 34.991359 ms 23/06/28 07:51:24 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:51:24 INFO DAGScheduler: Got job 62 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:51:24 INFO DAGScheduler: Final stage: ResultStage 157 (collect at Checksum.scala:561) 23/06/28 07:51:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 156) 23/06/28 07:51:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:24 INFO DAGScheduler: Submitting ResultStage 157 (MapPartitionsRDD[327] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:24 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 157 (MapPartitionsRDD[327] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Adding task set 157.0 with 1 tasks resource profile 0 23/06/28 07:51:24 INFO FairSchedulableBuilder: Added task set TaskSet_157.0 tasks to pool 974573833245257412 23/06/28 07:51:24 INFO TaskSetManager: Starting task 0.0 in stage 157.0 (TID 1325) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_102 stored as values in memory (estimated size 502.8 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_102_piece0 stored as bytes in memory (estimated size 126.7 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on 10.60.24.15:40507 (size: 126.7 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 102 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on 10.60.24.10:41041 (size: 126.7 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 37 to 10.60.24.10:44288 23/06/28 07:51:24 INFO BlockManagerInfo: Added rdd_323_0 in memory on 10.60.24.10:41041 (size: 1421.0 B, free: 487.7 GiB) 23/06/28 07:51:24 INFO TaskSetManager: Finished task 0.0 in stage 157.0 (TID 1325) in 63 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Removed TaskSet 157.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:24 INFO DAGScheduler: ResultStage 157 (collect at Checksum.scala:561) finished in 0.087 s 23/06/28 07:51:24 INFO DAGScheduler: Job 62 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:24 INFO TaskSchedulerImpl: Killing all running tasks in stage 157: Stage finished 23/06/28 07:51:24 INFO DAGScheduler: Job 62 finished: collect at Checksum.scala:561, took 0.088863 s 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:24 INFO DAGScheduler: Registering RDD 330 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 38 23/06/28 07:51:24 INFO DAGScheduler: Got map stage job 63 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:24 INFO DAGScheduler: Final stage: ShuffleMapStage 159 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 158) 23/06/28 07:51:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:24 INFO DAGScheduler: Submitting ShuffleMapStage 159 (MapPartitionsRDD[330] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:24 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 159 (MapPartitionsRDD[330] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Adding task set 159.0 with 1 tasks resource profile 0 23/06/28 07:51:24 INFO FairSchedulableBuilder: Added task set TaskSet_159.0 tasks to pool 974573833245257412 23/06/28 07:51:24 INFO TaskSetManager: Starting task 0.0 in stage 159.0 (TID 1326) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_103 stored as values in memory (estimated size 473.2 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_103_piece0 stored as bytes in memory (estimated size 120.8 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on 10.60.24.15:40507 (size: 120.8 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 103 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on 10.60.24.10:41041 (size: 120.8 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO TaskSetManager: Finished task 0.0 in stage 159.0 (TID 1326) in 31 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Removed TaskSet 159.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:24 INFO DAGScheduler: ShuffleMapStage 159 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.040 s 23/06/28 07:51:24 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:24 INFO DAGScheduler: running: Set() 23/06/28 07:51:24 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:24 INFO DAGScheduler: failed: Set() 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:24 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:51:24 INFO DAGScheduler: Got job 64 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:51:24 INFO DAGScheduler: Final stage: ResultStage 162 (first at Snapshot.scala:271) 23/06/28 07:51:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 161) 23/06/28 07:51:24 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:24 INFO DAGScheduler: Submitting ResultStage 162 (MapPartitionsRDD[332] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:24 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:24 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:24 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:24 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 162 (MapPartitionsRDD[332] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Adding task set 162.0 with 1 tasks resource profile 0 23/06/28 07:51:24 INFO FairSchedulableBuilder: Added task set TaskSet_162.0 tasks to pool 974573833245257412 23/06/28 07:51:24 INFO TaskSetManager: Starting task 0.0 in stage 162.0 (TID 1327) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_104 stored as values in memory (estimated size 419.4 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO MemoryStore: Block broadcast_104_piece0 stored as bytes in memory (estimated size 110.5 KiB, free 750.6 GiB) 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on 10.60.24.15:40507 (size: 110.5 KiB, free: 750.7 GiB) 23/06/28 07:51:24 INFO SparkContext: Created broadcast 104 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:24 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on 10.60.24.13:44035 (size: 110.5 KiB, free: 487.7 GiB) 23/06/28 07:51:24 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 38 to 10.60.24.13:58548 23/06/28 07:51:24 INFO TaskSetManager: Finished task 0.0 in stage 162.0 (TID 1327) in 42 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:24 INFO TaskSchedulerImpl: Removed TaskSet 162.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:24 INFO DAGScheduler: ResultStage 162 (first at Snapshot.scala:271) finished in 0.055 s 23/06/28 07:51:24 INFO DAGScheduler: Job 64 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:24 INFO TaskSchedulerImpl: Killing all running tasks in stage 162: Stage finished 23/06/28 07:51:24 INFO DAGScheduler: Job 64 finished: first at Snapshot.scala:271, took 0.055841 s 23/06/28 07:51:24 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:24 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log, version=61, metadata=Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)), logSegment=LogSegment(dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log,61,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json; isDirectory=false; length=1943; replication=1; blocksize=268435456; modification_time=1687938684000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC785C04B5"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000060.checkpoint.parquet; isDirectory=false; length=29625; replication=1; blocksize=268435456; modification_time=1687937289000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A938E70CD4'}),Some(60),1687938684000), checksumOpt=Some(VersionChecksum(Some(b6dd64e0-f25a-4992-9308-e68b3ea1b093),86107,6,None,None,1,1,Some(Stream()),Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@49d14876,[J@27dbabc)),None,Some(Stream(AddFile(part-00000-6b9b3a1d-a8da-45de-a4b3-8cd50f6ca176-c000.snappy.parquet,Map(),1392,1658882351000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-78d99c18-4585-493e-80ff-883714b186ca-c000.snappy.parquet,Map(),1392,1642649707000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-0321307b-64b5-4330-9ea7-f4a1546c9fc7-c000.snappy.parquet,Map(),1392,1658899177000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-47a3c79f-1f53-4b6a-80c4-bd71f2f757a8-c000.snappy.parquet,Map(),79147,1658899170000,false,{"numRecords":5752,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2022-01-20T14:30:28.955Z","_DLTrustedZoneTimeStamp":"2022-01-20T14:35:20.000Z","_RecordStart":"2022-01-20T14:35:20.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":0,"_RecordCurrent":1},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2022-07-26T18:00:15.455Z","_DLTrustedZoneTimeStamp":"2022-07-26T18:11:19.000Z","_RecordStart":"2022-07-26T18:10:50.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":1,"_RecordCurrent":1},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0,"_DLTrustedZoneTimeStamp":0,"_RecordStart":0,"_RecordEnd":0,"_RecordDeleted":0,"_RecordCurrent":0}},null,null), AddFile(part-00000-19db40fb-349f-41e5-af51-2ddf2517f348-c000.snappy.parquet,Map(),1392,1643247051000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-b3bce997-2d4c-43b0-94ba-039b842395a3-c000.snappy.parquet,Map(),1392,1642649740000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null)))))) 23/06/28 07:51:24 INFO MapPartitionsRDD: Removing RDD 246 from persistence list 23/06/28 07:51:24 INFO BlockManager: Removing RDD 246 23/06/28 07:51:24 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=b6dd64e0] Committed delta #61 to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log 23/06/28 07:51:24 INFO ChecksumHook: Writing checksum file for table path dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log version 61 23/06/28 07:51:24 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:51:24 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.crc using temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp 23/06/28 07:51:24 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp] Closing stream; size: 0 23/06/28 07:51:25 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp] Upload complete; size: 5149 23/06/28 07:51:25 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.crc] Starting rename. Issuing rename operation. 23/06/28 07:51:25 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.crc] Rename successful. 23/06/28 07:51:25 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000061.crc.f5802509-4688-4aef-8341-6b3dc096e380.tmp to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.crc 23/06/28 07:51:25 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:51:25 INFO ClusterLoadMonitor: Removed query with execution ID:165. Current active queries:0 23/06/28 07:51:25 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:25 INFO HiveMetaStore: 8: get_database: trusted 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:51:25 INFO HiveMetaStore: 8: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:25 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:25 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:51:25.131Z","run":{"runId":"f200eb6a-d0e3-4882-8df0-efcba40cff35","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.MergeIntoCommandEdge","num-children":0,"source":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"STG","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"merge_key","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2896,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, raw]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2938,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2939,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2940,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649443000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":false,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"target":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"MN","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2915,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2916,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2917,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2918,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"targetFileIndex":null,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2896,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],"matchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoMatchedUpdateClause","num-children":13,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"actions":[1,2,3,4,5,6,7,8,9,10,11,12]},{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2909,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2910,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2911,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Or","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2912,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2913,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Not","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2914,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2919,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2917,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2918,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2920,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"notMatchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoNotMatchedInsertClause","num-children":12,"actions":[0,1,2,3,4,5,6,7,8,9,10,11]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2931,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2932,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2933,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2934,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2935,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":2937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:14","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"notMatchedBySourceClauses":[],"migratedSchema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"trackHighWaterMarks":{"object":"scala.collection.immutable.Set$EmptySet$"}}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_merge_into_command_edge","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:25 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:25 INFO HiveMetaStore: 8: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO HiveMetaStore: 9: get_database: trusted 23/06/28 07:51:25 INFO HiveMetaStore: 10: get_database: raw 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:51:25 INFO HiveMetaStore: 9: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: raw 23/06/28 07:51:25 INFO HiveMetaStore: 10: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:51:25 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:25 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:25 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:51:25 INFO HiveMetaStore: 9: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:51:25 INFO HiveMetaStore: 10: get_multi_table : db=raw tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=raw tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:51:25 INFO ClusterLoadMonitor: Added query with execution ID:178. Current active queries:1 23/06/28 07:51:25 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:25 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:25 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:25.479Z","run":{"runId":"3538b3ba-923e-4a7d-8001-248d6c9fe335","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.MergeIntoCommandEdge","num-children":0,"source":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"STG","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"merge_key","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Join","num-children":2,"left":0,"right":1,"joinType":{"object":"org.apache.spark.sql.catalyst.plans.LeftOuter$"},"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"hint":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"TGT","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, raw]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4775,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4776,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4778,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4779,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4780,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4781,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4782,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4783,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4784,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649443000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":false,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"target":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"MN","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4741,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4742,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4743,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4744,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4745,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4746,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4747,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4748,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4749,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4750,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4751,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"targetFileIndex":null,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4743,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],"matchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoMatchedUpdateClause","num-children":13,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"actions":[1,2,3,4,5,6,7,8,9,10,11,12]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4749,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4750,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"notMatchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoNotMatchedInsertClause","num-children":12,"actions":[0,1,2,3,4,5,6,7,8,9,10,11]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"notMatchedBySourceClauses":[],"migratedSchema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"trackHighWaterMarks":{"object":"scala.collection.immutable.Set$EmptySet$"}}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_merge_into_command_edge","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:25 INFO SnapshotEdge: [tableId=5d0ae5b6-42a3-47d1-8d94-be2429376ace] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_105 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_105_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 105 from broadcast at DeltaLog.scala:634 23/06/28 07:51:25 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:25 INFO DAGScheduler: Got job 65 (executeCollect at DatasetRefCache.scala:71) with 1 output partitions 23/06/28 07:51:25 INFO DAGScheduler: Final stage: ResultStage 163 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:51:25 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:25 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:25 INFO DAGScheduler: Submitting ResultStage 163 (MapPartitionsRDD[335] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:51:25 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:25 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:25 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:25 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 163 (MapPartitionsRDD[335] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Adding task set 163.0 with 1 tasks resource profile 0 23/06/28 07:51:25 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 974573833245257412, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 974573833245257412. Created 974573833245257412 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:51:25 INFO FairSchedulableBuilder: Added task set TaskSet_163.0 tasks to pool 974573833245257412 23/06/28 07:51:25 INFO TaskSetManager: Starting task 0.0 in stage 163.0 (TID 1328) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_106 stored as values in memory (estimated size 209.5 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_106_piece0 stored as bytes in memory (estimated size 45.0 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on 10.60.24.15:40507 (size: 45.0 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 106 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on 10.60.24.10:41041 (size: 45.0 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 0.0 in stage 163.0 (TID 1328) in 45 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Removed TaskSet 163.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:25 INFO DAGScheduler: ResultStage 163 (executeCollect at DatasetRefCache.scala:71) finished in 0.052 s 23/06/28 07:51:25 INFO DAGScheduler: Job 65 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:25 INFO TaskSchedulerImpl: Killing all running tasks in stage 163: Stage finished 23/06/28 07:51:25 INFO DAGScheduler: Job 65 finished: executeCollect at DatasetRefCache.scala:71, took 0.053582 s 23/06/28 07:51:25 INFO PropagateEmptyRelation: Removed leaf nodes: - LocalRelation , [path#4928, partitionValues#4929, size#4930L, modificationTime#4931L, dataChange#4932, tags#4933, deletionVector#4934, stats_parsed#4935, version#4936L] 23/06/28 07:51:25 INFO FileSourceStrategy: Pushed Filters: IsNotNull(add.dataChange) 23/06/28 07:51:25 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(add#4849.dataChange) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_107 stored as values in memory (estimated size 600.8 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on 10.60.24.15:40507 (size: 16.4 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 107 from executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:25 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:25 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:51:25 INFO DAGScheduler: Got job 66 (executeCollect at DatasetRefCache.scala:71) with 1 output partitions 23/06/28 07:51:25 INFO DAGScheduler: Final stage: ResultStage 164 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:51:25 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:25 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:25 INFO DAGScheduler: Submitting ResultStage 164 (MapPartitionsRDD[338] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:51:25 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:25 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:25 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:25 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 164 (MapPartitionsRDD[338] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Adding task set 164.0 with 1 tasks resource profile 0 23/06/28 07:51:25 INFO FairSchedulableBuilder: Added task set TaskSet_164.0 tasks to pool 974573833245257412 23/06/28 07:51:25 INFO TaskSetManager: Starting task 0.0 in stage 164.0 (TID 1329) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_108 stored as values in memory (estimated size 260.9 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_108_piece0 stored as bytes in memory (estimated size 60.1 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on 10.60.24.15:40507 (size: 60.1 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 108 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on 10.60.24.10:41041 (size: 60.1 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on 10.60.24.10:41041 (size: 16.4 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 0.0 in stage 164.0 (TID 1329) in 43 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Removed TaskSet 164.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:25 INFO DAGScheduler: ResultStage 164 (executeCollect at DatasetRefCache.scala:71) finished in 0.045 s 23/06/28 07:51:25 INFO DAGScheduler: Job 66 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:25 INFO TaskSchedulerImpl: Killing all running tasks in stage 164: Stage finished 23/06/28 07:51:25 INFO DAGScheduler: Job 66 finished: executeCollect at DatasetRefCache.scala:71, took 0.046759 s 23/06/28 07:51:25 INFO CodeGenerator: Code generated in 8.999628 ms 23/06/28 07:51:25 INFO FileSourceStrategy: Pushed Filters: IsNotNull(_RecordCurrent),IsNotNull(_RecordDeleted),EqualTo(_RecordCurrent,1),EqualTo(_RecordDeleted,0) 23/06/28 07:51:25 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(_RecordCurrent#4764),isnotnull(_RecordDeleted#4763),(_RecordCurrent#4764 = 1),(_RecordDeleted#4763 = 0) 23/06/28 07:51:25 INFO DataSourceStrategy: Pruning directories with: 23/06/28 07:51:25 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:25 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:25 INFO CodeGenerator: Code generated in 7.61146 ms 23/06/28 07:51:25 INFO CodeGenerator: Code generated in 7.780936 ms 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_109 stored as values in memory (estimated size 535.4 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_110 stored as values in memory (estimated size 551.4 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_109_piece0 stored as bytes in memory (estimated size 15.0 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on 10.60.24.15:40507 (size: 15.0 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 109 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on 10.60.24.15:40507 (size: 15.3 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 110 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:25 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:25 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:25 INFO DAGScheduler: Registering RDD 342 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 39 23/06/28 07:51:25 INFO DAGScheduler: Got map stage job 67 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:25 INFO DAGScheduler: Final stage: ShuffleMapStage 165 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:25 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:25 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:25 INFO DAGScheduler: Submitting ShuffleMapStage 165 (MapPartitionsRDD[342] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:25 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:25 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:25 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:25 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 165 (MapPartitionsRDD[342] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Adding task set 165.0 with 1 tasks resource profile 0 23/06/28 07:51:25 INFO FairSchedulableBuilder: Added task set TaskSet_165.0 tasks to pool 974573833245257412 23/06/28 07:51:25 INFO DAGScheduler: Registering RDD 346 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 40 23/06/28 07:51:25 INFO DAGScheduler: Got map stage job 68 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:25 INFO DAGScheduler: Final stage: ShuffleMapStage 166 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:25 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:25 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:25 INFO TaskSetManager: Starting task 0.0 in stage 165.0 (TID 1330) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO DAGScheduler: Submitting ShuffleMapStage 166 (MapPartitionsRDD[346] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_111 stored as values in memory (estimated size 189.7 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_111_piece0 stored as bytes in memory (estimated size 48.9 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on 10.60.24.15:40507 (size: 48.9 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 111 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:25 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:25 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:25 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 166 (MapPartitionsRDD[346] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Adding task set 166.0 with 6 tasks resource profile 0 23/06/28 07:51:25 INFO FairSchedulableBuilder: Added task set TaskSet_166.0 tasks to pool 974573833245257412 23/06/28 07:51:25 INFO TaskSetManager: Starting task 0.0 in stage 166.0 (TID 1331) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO TaskSetManager: Starting task 4.0 in stage 166.0 (TID 1332) (10.60.24.10, executor 0, partition 4, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO TaskSetManager: Starting task 1.0 in stage 166.0 (TID 1333) (10.60.24.13, executor 1, partition 1, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO TaskSetManager: Starting task 5.0 in stage 166.0 (TID 1334) (10.60.24.10, executor 0, partition 5, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO TaskSetManager: Starting task 2.0 in stage 166.0 (TID 1335) (10.60.24.13, executor 1, partition 2, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO TaskSetManager: Starting task 3.0 in stage 166.0 (TID 1336) (10.60.24.13, executor 1, partition 3, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_112 stored as values in memory (estimated size 195.8 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO MemoryStore: Block broadcast_112_piece0 stored as bytes in memory (estimated size 50.2 KiB, free 750.6 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on 10.60.24.15:40507 (size: 50.2 KiB, free: 750.7 GiB) 23/06/28 07:51:25 INFO SparkContext: Created broadcast 112 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on 10.60.24.13:44035 (size: 48.9 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on 10.60.24.13:44035 (size: 50.2 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on 10.60.24.10:41041 (size: 50.2 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on 10.60.24.10:41041 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on 10.60.24.13:44035 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 5.0 in stage 166.0 (TID 1334) in 39 ms on 10.60.24.10 (executor 0) (1/6) 23/06/28 07:51:25 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on 10.60.24.13:44035 (size: 15.0 KiB, free: 487.7 GiB) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 4.0 in stage 166.0 (TID 1332) in 70 ms on 10.60.24.10 (executor 0) (2/6) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 1.0 in stage 166.0 (TID 1333) in 89 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 3.0 in stage 166.0 (TID 1336) in 88 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 0.0 in stage 166.0 (TID 1331) in 90 ms on 10.60.24.13 (executor 1) (5/6) 23/06/28 07:51:25 INFO TaskSetManager: Finished task 2.0 in stage 166.0 (TID 1335) in 89 ms on 10.60.24.13 (executor 1) (6/6) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Removed TaskSet 166.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:25 INFO DAGScheduler: ShuffleMapStage 166 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.095 s 23/06/28 07:51:25 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:25 INFO DAGScheduler: running: Set(ShuffleMapStage 165) 23/06/28 07:51:25 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:25 INFO DAGScheduler: failed: Set() 23/06/28 07:51:25 INFO TaskSetManager: Finished task 0.0 in stage 165.0 (TID 1330) in 116 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:25 INFO TaskSchedulerImpl: Removed TaskSet 165.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:25 INFO DAGScheduler: ShuffleMapStage 165 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.120 s 23/06/28 07:51:25 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:25 INFO DAGScheduler: running: Set() 23/06/28 07:51:25 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:25 INFO DAGScheduler: failed: Set() 23/06/28 07:51:26 INFO ShufflePartitionsUtil: For shuffle(39), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:26 INFO DAGScheduler: Registering RDD 348 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 41 23/06/28 07:51:26 INFO DAGScheduler: Got map stage job 69 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ShuffleMapStage 168 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 167) 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ShuffleMapStage 168 (MapPartitionsRDD[348] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 168 (MapPartitionsRDD[348] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 168.0 with 1 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_168.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 168.0 (TID 1337) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_113 stored as values in memory (estimated size 128.0 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_113_piece0 stored as bytes in memory (estimated size 20.8 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on 10.60.24.15:40507 (size: 20.8 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 113 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on 10.60.24.13:44035 (size: 20.8 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 39 to 10.60.24.13:58548 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 168.0 (TID 1337) in 19 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 168.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ShuffleMapStage 168 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.022 s 23/06/28 07:51:26 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:26 INFO DAGScheduler: running: Set() 23/06/28 07:51:26 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:26 INFO DAGScheduler: failed: Set() 23/06/28 07:51:26 INFO ShufflePartitionsUtil: For shuffle(40), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:26 INFO CodeGenerator: Code generated in 11.977458 ms 23/06/28 07:51:26 INFO SparkEBJCleanupListener: Cleaning up ebj join relation block with ID spark_join_relation_41_1340350209) 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:26 WARN BlockManager: Asked to remove block spark_join_relation_41_1340350209, which does not exist 23/06/28 07:51:26 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":89197,"lifetime":384598},"10.60.24.13":{"diskUsage":74852,"lifetime":384868}}, ioCacheNumScanTasks={"numLocalScanTasks":26,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_114 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 114 from broadcast at BitmapFilter.scala:230 23/06/28 07:51:26 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:26 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:26 INFO CodeGenerator: Code generated in 6.4474 ms 23/06/28 07:51:26 INFO CodeGenerator: Code generated in 6.988142 ms 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_115 stored as values in memory (estimated size 539.9 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO DAGScheduler: Registering RDD 358 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 42 23/06/28 07:51:26 INFO DAGScheduler: Got map stage job 70 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ShuffleMapStage 172 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 171, ShuffleMapStage 170) 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ShuffleMapStage 172 (MapPartitionsRDD[358] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_115_piece0 stored as bytes in memory (estimated size 15.1 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on 10.60.24.15:40507 (size: 15.1 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 115 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 172 (MapPartitionsRDD[358] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 172.0 with 6 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_172.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 172.0 (TID 1338) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 4.0 in stage 172.0 (TID 1339) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 1.0 in stage 172.0 (TID 1340) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 5.0 in stage 172.0 (TID 1341) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 2.0 in stage 172.0 (TID 1342) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 3.0 in stage 172.0 (TID 1343) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_116 stored as values in memory (estimated size 237.0 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in memory (estimated size 69.6 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on 10.60.24.15:40507 (size: 69.6 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 116 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on 10.60.24.13:44035 (size: 69.6 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on 10.60.24.10:41041 (size: 69.6 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO DAGScheduler: Registering RDD 362 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 43 23/06/28 07:51:26 INFO DAGScheduler: Got map stage job 71 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ShuffleMapStage 173 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ShuffleMapStage 173 (MapPartitionsRDD[362] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 173 (MapPartitionsRDD[362] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 173.0 with 6 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_173.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 173.0 (TID 1344) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 4.0 in stage 173.0 (TID 1345) (10.60.24.10, executor 0, partition 4, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 1.0 in stage 173.0 (TID 1346) (10.60.24.13, executor 1, partition 1, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 5.0 in stage 173.0 (TID 1347) (10.60.24.10, executor 0, partition 5, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 2.0 in stage 173.0 (TID 1348) (10.60.24.13, executor 1, partition 2, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 3.0 in stage 173.0 (TID 1349) (10.60.24.13, executor 1, partition 3, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_117 stored as values in memory (estimated size 192.7 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in memory (estimated size 49.8 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on 10.60.24.15:40507 (size: 49.8 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 117 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on 10.60.24.10:41041 (size: 49.8 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on 10.60.24.13:44035 (size: 49.8 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 40 to 10.60.24.10:44288 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 41 to 10.60.24.10:44288 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on 10.60.24.13:44035 (size: 15.1 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 173.0 (TID 1344) in 50 ms on 10.60.24.13 (executor 1) (1/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 2.0 in stage 173.0 (TID 1348) in 50 ms on 10.60.24.13 (executor 1) (2/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 1.0 in stage 173.0 (TID 1346) in 50 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 3.0 in stage 173.0 (TID 1349) in 50 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on 10.60.24.10:41041 (size: 15.1 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 5.0 in stage 173.0 (TID 1347) in 69 ms on 10.60.24.10 (executor 0) (5/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 4.0 in stage 173.0 (TID 1345) in 98 ms on 10.60.24.10 (executor 0) (6/6) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 173.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ShuffleMapStage 173 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.103 s 23/06/28 07:51:26 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:26 INFO DAGScheduler: running: Set(ShuffleMapStage 172) 23/06/28 07:51:26 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:26 INFO DAGScheduler: failed: Set() 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:26 INFO DAGScheduler: Asked to cancel job 70 23/06/28 07:51:26 INFO TaskSchedulerImpl: Cancelling stage 172 23/06/28 07:51:26 INFO TaskSchedulerImpl: Killing all running tasks in stage 172: Stage cancelled 23/06/28 07:51:26 INFO ShufflePartitionsUtil: For shuffle(43), advisory target size: 67108864, actual target size 1048576, minimum partition size: 1048576 23/06/28 07:51:26 INFO TaskSchedulerImpl: Stage 172 was cancelled 23/06/28 07:51:26 INFO DAGScheduler: ShuffleMapStage 172 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) failed in 0.143 s due to Job 70 cancelled 23/06/28 07:51:26 INFO DAGScheduler: Registering RDD 364 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 44 23/06/28 07:51:26 INFO DAGScheduler: Got map stage job 72 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ShuffleMapStage 175 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 174) 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ShuffleMapStage 175 (MapPartitionsRDD[364] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 175 (MapPartitionsRDD[364] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 175.0 with 6 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_175.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 4.0 in stage 175.0 (TID 1350) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 175.0 (TID 1351) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 5.0 in stage 175.0 (TID 1352) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 1.0 in stage 175.0 (TID 1353) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 2.0 in stage 175.0 (TID 1354) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 3.0 in stage 175.0 (TID 1355) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_118 stored as values in memory (estimated size 128.4 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in memory (estimated size 21.1 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on 10.60.24.15:40507 (size: 21.1 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 118 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on 10.60.24.10:41041 (size: 21.1 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on 10.60.24.13:44035 (size: 21.1 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 43 to 10.60.24.10:44288 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 43 to 10.60.24.13:58548 23/06/28 07:51:26 WARN TaskSetManager: Lost task 5.0 in stage 172.0 (TID 1341) (10.60.24.10 executor 0): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 WARN TaskSetManager: Lost task 2.0 in stage 172.0 (TID 1342) (10.60.24.13 executor 1): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 WARN TaskSetManager: Lost task 0.0 in stage 172.0 (TID 1338) (10.60.24.13 executor 1): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 WARN TaskSetManager: Lost task 3.0 in stage 172.0 (TID 1343) (10.60.24.13 executor 1): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 WARN TaskSetManager: Lost task 4.0 in stage 172.0 (TID 1339) (10.60.24.10 executor 0): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Finished task 5.0 in stage 175.0 (TID 1352) in 17 ms on 10.60.24.10 (executor 0) (1/6) 23/06/28 07:51:26 WARN TaskSetManager: Lost task 1.0 in stage 172.0 (TID 1340) (10.60.24.13 executor 1): TaskKilled (Stage cancelled) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 172.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Finished task 1.0 in stage 175.0 (TID 1353) in 17 ms on 10.60.24.13 (executor 1) (2/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 175.0 (TID 1351) in 17 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 3.0 in stage 175.0 (TID 1355) in 17 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 2.0 in stage 175.0 (TID 1354) in 17 ms on 10.60.24.13 (executor 1) (5/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 4.0 in stage 175.0 (TID 1350) in 17 ms on 10.60.24.10 (executor 0) (6/6) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 175.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ShuffleMapStage 175 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.020 s 23/06/28 07:51:26 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:26 INFO DAGScheduler: running: Set() 23/06/28 07:51:26 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:26 INFO DAGScheduler: failed: Set() 23/06/28 07:51:26 INFO CodeGenerator: Code generated in 8.698897 ms 23/06/28 07:51:26 INFO DAGScheduler: Registering RDD 372 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 45 23/06/28 07:51:26 INFO DAGScheduler: Got map stage job 73 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 6 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ShuffleMapStage 181 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 179, ShuffleMapStage 180, ShuffleMapStage 178) 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ShuffleMapStage 181 (MapPartitionsRDD[372] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 181 (MapPartitionsRDD[372] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 181.0 with 6 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_181.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 181.0 (TID 1356) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 4.0 in stage 181.0 (TID 1357) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 1.0 in stage 181.0 (TID 1358) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 5.0 in stage 181.0 (TID 1359) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 2.0 in stage 181.0 (TID 1360) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 3.0 in stage 181.0 (TID 1361) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_119 stored as values in memory (estimated size 268.2 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_119_piece0 stored as bytes in memory (estimated size 84.5 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_119_piece0 in memory on 10.60.24.15:40507 (size: 84.5 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 119 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_119_piece0 in memory on 10.60.24.10:41041 (size: 84.5 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_119_piece0 in memory on 10.60.24.13:44035 (size: 84.5 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 40 to 10.60.24.10:44288 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 41 to 10.60.24.10:44288 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 40 to 10.60.24.13:58548 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 41 to 10.60.24.13:58548 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_5 on disk on 10.60.24.10:41041 (size: 46.0 B) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 44 to 10.60.24.10:44288 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_4 on disk on 10.60.24.10:41041 (size: 46.0 B) 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_1 on disk on 10.60.24.13:44035 (size: 46.0 B) 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_3 on disk on 10.60.24.13:44035 (size: 46.0 B) 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_0 on disk on 10.60.24.13:44035 (size: 46.0 B) 23/06/28 07:51:26 INFO BlockManagerInfo: Added rdd_356_2 on disk on 10.60.24.13:44035 (size: 46.0 B) 23/06/28 07:51:26 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 44 to 10.60.24.13:58548 23/06/28 07:51:26 INFO TaskSetManager: Finished task 5.0 in stage 181.0 (TID 1359) in 131 ms on 10.60.24.10 (executor 0) (1/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 4.0 in stage 181.0 (TID 1357) in 131 ms on 10.60.24.10 (executor 0) (2/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 1.0 in stage 181.0 (TID 1358) in 139 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 2.0 in stage 181.0 (TID 1360) in 139 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 181.0 (TID 1356) in 139 ms on 10.60.24.13 (executor 1) (5/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 3.0 in stage 181.0 (TID 1361) in 139 ms on 10.60.24.13 (executor 1) (6/6) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 181.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ShuffleMapStage 181 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.150 s 23/06/28 07:51:26 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:26 INFO DAGScheduler: running: Set() 23/06/28 07:51:26 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:26 INFO DAGScheduler: failed: Set() 23/06/28 07:51:26 INFO AQEPropagateEmptyRelation: Removed leaf nodes: - LogicalQueryStage Aggregate [fileName#5113, null], [fileName#5113, count(1) AS secondaryRowCount#5126L, bitmapaggregator(primaryRowIndex#5116L, 0, 0, Portable) AS CardinalityAndBitmapStructConditional#5128, max(primaryRowIndex#5116L) AS maxPrimaryRowIndexCol#5130L, bitmapaggregator(secondaryRowIndex#5115L, 0, 0, Portable) AS CardinalityAndBitmapStruct#5132, count(distinct secondaryRowIndex#5115L) AS secondaryRowIndexCardinality#5133L], ObjectHashAggregate(keys=[fileName#5113, null#5291], functions=[finalmerge_count(merge count#5182L) AS count(1)#5125L, finalmerge_bitmapaggregator(merge buf#5184) AS bitmapaggregator(primaryRowIndex#5116L, 0, 0, Portable)#5127, finalmerge_max(merge max#5186L) AS max(primaryRowIndex#5116L)#5129L, finalmerge_bitmapaggregator(merge buf#5188) AS bitmapaggregator(secondaryRowIndex#5115L, 0, 0, Portable)#5131, finalmerge_count(distinct merge count#5190L) AS count(secondaryRowIndex#5115L)#5134L]) 23/06/28 07:51:26 INFO AQEPropagateEmptyRelation: Removed leaf nodes: - LocalRelation , [fileName#5113, secondaryRowCount#5126L, CardinalityAndBitmapStructConditional#5128, maxPrimaryRowIndexCol#5130L, CardinalityAndBitmapStruct#5132, secondaryRowIndexCardinality#5133L] 23/06/28 07:51:26 INFO SparkContext: Starting job: collect at LowShuffleMergeExecutor.scala:638 23/06/28 07:51:26 INFO DAGScheduler: Job 74 finished: collect at LowShuffleMergeExecutor.scala:638, took 0.000175 s 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:26 INFO SQLExecution: Received onJobStart for query with unregistered executionId 182 23/06/28 07:51:26 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:26 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":89197,"lifetime":385084},"10.60.24.13":{"diskUsage":74852,"lifetime":385354}}, ioCacheNumScanTasks={"numLocalScanTasks":32,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:26 INFO FileSizeAutoTuner: Rewrite-heavy auto-detection result in 170 ms: {"isRewriteHeavy":true,"numRewriteOpsFound":10,"numDataChangeOpsFound":10,"numNonDataChangeOpsFound":0,"autoDetectNumHistoryOps":10,"autoDetectRewriteOpsFraction":0.9} 23/06/28 07:51:26 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"rewrites","tunedConfs":{"spark.databricks.delta.optimize.maxFileSize":"67108864","spark.databricks.delta.optimize.minFileSize":"33554432","spark.databricks.delta.autoCompact.minFileSize":"8388608","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimizeWrite.binSize":"192"}} 23/06/28 07:51:26 INFO ParquetUtils: Using user defined output committer for Parquet: org.apache.spark.sql.parquet.DirectParquetOutputCommitter 23/06/28 07:51:26 INFO CodeGenerator: Code generated in 7.347115 ms 23/06/28 07:51:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:26 INFO SparkContext: Starting job: write at WriteIntoDeltaCommand.scala:70 23/06/28 07:51:26 INFO DAGScheduler: Got job 75 (write at WriteIntoDeltaCommand.scala:70) with 6 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ResultStage 185 (write at WriteIntoDeltaCommand.scala:70) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 183, ShuffleMapStage 184) 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ResultStage 185 (MapPartitionsRDD[378] at write at WriteIntoDeltaCommand.scala:70), which has no missing parents 23/06/28 07:51:26 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:26.785Z","run":{"runId":"1e499ce2-ec52-4782-a1ff-e46a0762ae0d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[component, groupid, id, itemid, timeadded, userid, _DLRawZoneTimeStamp, _DLTrustedZoneTimeStamp, _RecordStart, _RecordEnd, _RecordDeleted, _RecordCurrent]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5329,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLTrustedZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5336,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordStart","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5337,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordEnd","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5338,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordDeleted","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5339,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordCurrent","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5340,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"true","dataType":"boolean"}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null,"_originStats":null,"_originConstraints":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/year=2023/month=06/day=28","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"merge_key","type":"string"},{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"}]}},"inputFacets":{}},{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"merge_key","type":"string"},{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 185 (MapPartitionsRDD[378] at write at WriteIntoDeltaCommand.scala:70) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 185.0 with 6 tasks resource profile 0 23/06/28 07:51:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_185.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 4.0 in stage 185.0 (TID 1362) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 185.0 (TID 1363) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 5.0 in stage 185.0 (TID 1364) (10.60.24.10, executor 0, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 1.0 in stage 185.0 (TID 1365) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 2.0 in stage 185.0 (TID 1366) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO TaskSetManager: Starting task 3.0 in stage 185.0 (TID 1367) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_120 stored as values in memory (estimated size 349.6 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_120_piece0 stored as bytes in memory (estimated size 118.4 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_120_piece0 in memory on 10.60.24.15:40507 (size: 118.4 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 120 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:51:26 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:51:26 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:51:26 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_120_piece0 in memory on 10.60.24.13:44035 (size: 118.4 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_120_piece0 in memory on 10.60.24.10:41041 (size: 118.4 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 185.0 (TID 1363) in 84 ms on 10.60.24.13 (executor 1) (1/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 1.0 in stage 185.0 (TID 1365) in 84 ms on 10.60.24.13 (executor 1) (2/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 2.0 in stage 185.0 (TID 1366) in 84 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 3.0 in stage 185.0 (TID 1367) in 89 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 5.0 in stage 185.0 (TID 1364) in 108 ms on 10.60.24.10 (executor 0) (5/6) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 4.0 in stage 185.0 (TID 1362) in 109 ms on 10.60.24.10 (executor 0) (6/6) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 185.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ResultStage 185 (write at WriteIntoDeltaCommand.scala:70) finished in 0.117 s 23/06/28 07:51:26 INFO DAGScheduler: Job 75 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:26 INFO TaskSchedulerImpl: Killing all running tasks in stage 185: Stage finished 23/06/28 07:51:26 INFO DAGScheduler: Job 75 finished: write at WriteIntoDeltaCommand.scala:70, took 0.119296 s 23/06/28 07:51:26 INFO FileFormatWriter: Start to commit write Job 166c72da-9e39-4678-a591-178aa042524d. 23/06/28 07:51:26 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:51:26 INFO FileFormatWriter: Write Job 166c72da-9e39-4678-a591-178aa042524d committed. Elapsed time: 1 ms. 23/06/28 07:51:26 INFO FileFormatWriter: Finished processing stats for write job 166c72da-9e39-4678-a591-178aa042524d. 23/06/28 07:51:26 INFO SparkContext: Starting job: collect at TransactionalWriteEdge.scala:576 23/06/28 07:51:26 INFO DAGScheduler: Got job 76 (collect at TransactionalWriteEdge.scala:576) with 1 output partitions 23/06/28 07:51:26 INFO DAGScheduler: Final stage: ResultStage 186 (collect at TransactionalWriteEdge.scala:576) 23/06/28 07:51:26 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:26 INFO DAGScheduler: Submitting ResultStage 186 (SQLExecutionRDD[381] at toRdd at TransactionalWriteEdge.scala:576), which has no missing parents 23/06/28 07:51:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:26 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 186 (SQLExecutionRDD[381] at toRdd at TransactionalWriteEdge.scala:576) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Adding task set 186.0 with 1 tasks resource profile 0 23/06/28 07:51:26 INFO FairSchedulableBuilder: Added task set TaskSet_186.0 tasks to pool 974573833245257412 23/06/28 07:51:26 INFO TaskSetManager: Starting task 0.0 in stage 186.0 (TID 1368) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_121 stored as values in memory (estimated size 148.8 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO MemoryStore: Block broadcast_121_piece0 stored as bytes in memory (estimated size 30.1 KiB, free 750.6 GiB) 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_121_piece0 in memory on 10.60.24.15:40507 (size: 30.1 KiB, free: 750.7 GiB) 23/06/28 07:51:26 INFO SparkContext: Created broadcast 121 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:26 INFO BlockManagerInfo: Added broadcast_121_piece0 in memory on 10.60.24.13:44035 (size: 30.1 KiB, free: 487.7 GiB) 23/06/28 07:51:26 INFO TaskSetManager: Finished task 0.0 in stage 186.0 (TID 1368) in 14 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:26 INFO TaskSchedulerImpl: Removed TaskSet 186.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:26 INFO DAGScheduler: ResultStage 186 (collect at TransactionalWriteEdge.scala:576) finished in 0.016 s 23/06/28 07:51:26 INFO DAGScheduler: Job 76 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:26 INFO TaskSchedulerImpl: Killing all running tasks in stage 186: Stage finished 23/06/28 07:51:26 INFO DAGScheduler: Job 76 finished: collect at TransactionalWriteEdge.scala:576, took 0.017175 s 23/06/28 07:51:26 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=ae7c3139] Attempting to commit version 62 with 1 actions with WriteSerializable isolation level 23/06/28 07:51:26 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:51:27 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json using temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp 23/06/28 07:51:27 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp] Closing stream; size: 0 23/06/28 07:51:27 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp] Upload complete; size: 1364 23/06/28 07:51:27 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json] Starting rename. Issuing rename operation. 23/06/28 07:51:27 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json] Rename successful. 23/06/28 07:51:27 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.json.7212eda9-482c-4712-a2ea-2db26c3003ac.tmp to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json 23/06/28 07:51:27 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=ae7c3139] Incremental commit: starting with snapshot version 61 23/06/28 07:51:27 INFO DeltaLog: Loading version 62 starting from checkpoint version 60. 23/06/28 07:51:27 INFO SnapshotEdge: [tableId=5d0ae5b6-42a3-47d1-8d94-be2429376ace] Created snapshot SnapshotEdge(path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log, version=62, metadata=Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)), logSegment=LogSegment(dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log,62,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json; isDirectory=false; length=1943; replication=1; blocksize=268435456; modification_time=1687938684000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC785C04B5"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json; isDirectory=false; length=1364; replication=1; blocksize=268435456; modification_time=1687938687000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC7A258EF7"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000060.checkpoint.parquet; isDirectory=false; length=29625; replication=1; blocksize=268435456; modification_time=1687937289000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A938E70CD4'}),Some(60),1687938687000), checksumOpt=Some(VersionChecksum(Some(ae7c3139-06d4-4bc6-a1df-05c133e75a24),86107,6,None,None,1,1,Some(Stream()),Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@558ef978,[J@68ffc89c)),None,Some(Stream(AddFile(part-00000-6b9b3a1d-a8da-45de-a4b3-8cd50f6ca176-c000.snappy.parquet,Map(),1392,1658882351000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-0321307b-64b5-4330-9ea7-f4a1546c9fc7-c000.snappy.parquet,Map(),1392,1658899177000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-78d99c18-4585-493e-80ff-883714b186ca-c000.snappy.parquet,Map(),1392,1642649707000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-47a3c79f-1f53-4b6a-80c4-bd71f2f757a8-c000.snappy.parquet,Map(),79147,1658899170000,false,{"numRecords":5752,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2022-01-20T14:30:28.955Z","_DLTrustedZoneTimeStamp":"2022-01-20T14:35:20.000Z","_RecordStart":"2022-01-20T14:35:20.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":0,"_RecordCurrent":1},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2022-07-26T18:00:15.455Z","_DLTrustedZoneTimeStamp":"2022-07-26T18:11:19.000Z","_RecordStart":"2022-07-26T18:10:50.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":1,"_RecordCurrent":1},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0,"_DLTrustedZoneTimeStamp":0,"_RecordStart":0,"_RecordEnd":0,"_RecordDeleted":0,"_RecordCurrent":0}},null,null), AddFile(part-00000-19db40fb-349f-41e5-af51-2ddf2517f348-c000.snappy.parquet,Map(),1392,1643247051000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-b3bce997-2d4c-43b0-94ba-039b842395a3-c000.snappy.parquet,Map(),1392,1642649740000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null)))))) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_122 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_122_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_122_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 122 from broadcast at DeltaLog.scala:634 23/06/28 07:51:27 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(Parquet, numFilesInSegment: 1, totalFileSize: 29625) 23/06/28 07:51:27 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 2, totalFileSize: 3307) 23/06/28 07:51:27 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:26.815Z","run":{"runId":"1e499ce2-ec52-4782-a1ff-e46a0762ae0d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[component, groupid, id, itemid, timeadded, userid, _DLRawZoneTimeStamp, _DLTrustedZoneTimeStamp, _RecordStart, _RecordEnd, _RecordDeleted, _RecordCurrent]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5329,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLTrustedZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5336,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordStart","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5337,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordEnd","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5338,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordDeleted","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5339,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordCurrent","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5340,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"true","dataType":"boolean"}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null,"_originStats":null,"_originConstraints":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532185","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/trusted/load-trusted-zone-v2","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"409724359447930","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-raw/lms/lmsuu_mdl_groups_members/delta/year=2023/month=06/day=28","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"merge_key","type":"string"},{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"}]}},"inputFacets":{}},{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"merge_key","type":"string"},{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:27 INFO SQLExecution: Received onJobStart for query with unregistered executionId 184 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on {} java.lang.NullPointerException at io.openlineage.spark.agent.util.ScalaConversionUtils.fromSeq(ScalaConversionUtils.java:61) at io.openlineage.spark.agent.lifecycle.Rdds.findFileLikeRdds(Rdds.java:69) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:36) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:26) at io.openlineage.spark.agent.util.PlanUtils.safeIsDefinedAt(PlanUtils.java:276) at io.openlineage.spark.agent.util.PlanUtils$1.lambda$isDefinedAt$0(PlanUtils.java:83) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531) at io.openlineage.spark.agent.util.PlanUtils$1.isDefinedAt(PlanUtils.java:84) at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:30) at scala.PartialFunction$AndThen.applyOrElse(PartialFunction.scala:194) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$visitLogicalPlan$14(OpenLineageRunEventBuilder.java:397) at io.openlineage.spark.agent.util.ScalaConversionUtils$3.apply(ScalaConversionUtils.java:131) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1$adapted(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:292) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.map(TreeNode.scala:312) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$buildInputDatasets$6(OpenLineageRunEventBuilder.java:348) at java.util.Optional.map(Optional.java:215) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildInputDatasets(OpenLineageRunEventBuilder.java:346) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.populateRun(OpenLineageRunEventBuilder.java:302) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:286) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:264) at io.openlineage.spark.agent.lifecycle.SparkSQLExecutionContext.end(SparkSQLExecutionContext.java:194) at io.openlineage.spark.agent.OpenLineageSparkListener.onJobEnd(OpenLineageSparkListener.java:167) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:39) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:118) at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:102) at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:114) at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:114) at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:109) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:105) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1655) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:105) 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:27 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:51:26.933Z","run":{"runId":"1e499ce2-ec52-4782-a1ff-e46a0762ae0d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[component, groupid, id, itemid, timeadded, userid, _DLRawZoneTimeStamp, _DLTrustedZoneTimeStamp, _RecordStart, _RecordEnd, _RecordDeleted, _RecordCurrent]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5329,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLTrustedZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5336,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordStart","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5337,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordEnd","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5338,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordDeleted","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5339,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordCurrent","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5340,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"true","dataType":"boolean"}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null,"_originStats":null,"_originConstraints":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on {} java.lang.NullPointerException at io.openlineage.spark.agent.util.ScalaConversionUtils.fromSeq(ScalaConversionUtils.java:61) at io.openlineage.spark.agent.lifecycle.Rdds.findFileLikeRdds(Rdds.java:69) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:36) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:26) at io.openlineage.spark.agent.util.PlanUtils.safeIsDefinedAt(PlanUtils.java:276) at io.openlineage.spark.agent.util.PlanUtils$1.lambda$isDefinedAt$0(PlanUtils.java:83) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531) at io.openlineage.spark.agent.util.PlanUtils$1.isDefinedAt(PlanUtils.java:84) at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:30) at scala.PartialFunction$AndThen.applyOrElse(PartialFunction.scala:194) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$visitLogicalPlan$14(OpenLineageRunEventBuilder.java:397) at io.openlineage.spark.agent.util.ScalaConversionUtils$3.apply(ScalaConversionUtils.java:131) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1$adapted(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:292) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.map(TreeNode.scala:312) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$buildInputDatasets$6(OpenLineageRunEventBuilder.java:348) at java.util.Optional.map(Optional.java:215) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildInputDatasets(OpenLineageRunEventBuilder.java:346) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.populateRun(OpenLineageRunEventBuilder.java:302) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:286) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:247) at io.openlineage.spark.agent.lifecycle.SparkSQLExecutionContext.start(SparkSQLExecutionContext.java:167) at io.openlineage.spark.agent.OpenLineageSparkListener.lambda$onJobStart$10(OpenLineageSparkListener.java:151) at java.util.Optional.ifPresent(Optional.java:159) at io.openlineage.spark.agent.OpenLineageSparkListener.onJobStart(OpenLineageSparkListener.java:147) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:37) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:118) at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:102) at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:114) at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:114) at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:109) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:105) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1655) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:105) 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:27 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:51:27 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:51:27 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:51:27 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:51:27 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:51:27 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:51:27 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:51:26.941Z","run":{"runId":"1e499ce2-ec52-4782-a1ff-e46a0762ae0d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[component, groupid, id, itemid, timeadded, userid, _DLRawZoneTimeStamp, _DLTrustedZoneTimeStamp, _RecordStart, _RecordEnd, _RecordDeleted, _RecordCurrent]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5329,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLTrustedZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5336,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordStart","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5337,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordEnd","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5338,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordDeleted","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5339,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordCurrent","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5340,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"true","dataType":"boolean"}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null,"_originStats":null,"_originConstraints":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532185","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/trusted/load-trusted-zone-v2","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"409724359447930","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:27 INFO SQLExecution: Received onJobStart for query with unregistered executionId 184 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on {} java.lang.NullPointerException at io.openlineage.spark.agent.util.ScalaConversionUtils.fromSeq(ScalaConversionUtils.java:61) at io.openlineage.spark.agent.lifecycle.Rdds.findFileLikeRdds(Rdds.java:69) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:36) at io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor.isDefinedAt(LogicalRDDVisitor.java:26) at io.openlineage.spark.agent.util.PlanUtils.safeIsDefinedAt(PlanUtils.java:276) at io.openlineage.spark.agent.util.PlanUtils$1.lambda$isDefinedAt$0(PlanUtils.java:83) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531) at io.openlineage.spark.agent.util.PlanUtils$1.isDefinedAt(PlanUtils.java:84) at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:30) at scala.PartialFunction$AndThen.applyOrElse(PartialFunction.scala:194) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$visitLogicalPlan$14(OpenLineageRunEventBuilder.java:397) at io.openlineage.spark.agent.util.ScalaConversionUtils$3.apply(ScalaConversionUtils.java:131) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$map$1$adapted(TreeNode.scala:312) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:292) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:293) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:293) at org.apache.spark.sql.catalyst.trees.TreeNode.map(TreeNode.scala:312) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$buildInputDatasets$6(OpenLineageRunEventBuilder.java:348) at java.util.Optional.map(Optional.java:215) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildInputDatasets(OpenLineageRunEventBuilder.java:346) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.populateRun(OpenLineageRunEventBuilder.java:302) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:286) at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:238) at io.openlineage.spark.agent.lifecycle.SparkSQLExecutionContext.end(SparkSQLExecutionContext.java:95) at io.openlineage.spark.agent.OpenLineageSparkListener.sparkSQLExecEnd(OpenLineageSparkListener.java:98) at io.openlineage.spark.agent.OpenLineageSparkListener.onOtherEvent(OpenLineageSparkListener.java:84) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:102) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:42) at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:118) at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:102) at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:114) at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:114) at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:109) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:105) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1655) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:105) 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:27 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:51:26.958Z","run":{"runId":"1e499ce2-ec52-4782-a1ff-e46a0762ae0d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[component, groupid, id, itemid, timeadded, userid, _DLRawZoneTimeStamp, _DLTrustedZoneTimeStamp, _RecordStart, _RecordEnd, _RecordDeleted, _RecordCurrent]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"component","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5329,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_DLTrustedZoneTimeStamp","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5336,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordStart","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5337,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordEnd","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5338,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordDeleted","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5339,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"_RecordCurrent","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5340,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"com.databricks.sql.execution.metric.IncrementMetric","num-children":1,"child":0,"metric":null},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"true","dataType":"boolean"}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null,"_originStats":null,"_originConstraints":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_123 stored as values in memory (estimated size 610.3 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_123_piece0 stored as bytes in memory (estimated size 17.5 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_123_piece0 in memory on 10.60.24.15:40507 (size: 17.5 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 123 from $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173 23/06/28 07:51:27 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:51:27 INFO DAGScheduler: Registering RDD 388 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 46 23/06/28 07:51:27 INFO DAGScheduler: Got map stage job 77 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 3 output partitions 23/06/28 07:51:27 INFO DAGScheduler: Final stage: ShuffleMapStage 187 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:27 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:51:27 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:27 INFO DAGScheduler: Submitting ShuffleMapStage 187 (MapPartitionsRDD[388] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:27 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:27 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:27 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:27 INFO DAGScheduler: Submitting 3 missing tasks from ShuffleMapStage 187 (MapPartitionsRDD[388] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0, 1, 2)) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Adding task set 187.0 with 3 tasks resource profile 0 23/06/28 07:51:27 INFO FairSchedulableBuilder: Added task set TaskSet_187.0 tasks to pool 974573833245257412 23/06/28 07:51:27 INFO TaskSetManager: Starting task 0.0 in stage 187.0 (TID 1369) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO TaskSetManager: Starting task 1.0 in stage 187.0 (TID 1370) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO TaskSetManager: Starting task 2.0 in stage 187.0 (TID 1371) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_124 stored as values in memory (estimated size 324.4 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_124_piece0 stored as bytes in memory (estimated size 79.5 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_124_piece0 in memory on 10.60.24.15:40507 (size: 79.5 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 124 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_124_piece0 in memory on 10.60.24.13:44035 (size: 79.5 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_124_piece0 in memory on 10.60.24.10:41041 (size: 79.5 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO TaskSetManager: Finished task 1.0 in stage 187.0 (TID 1370) in 17 ms on 10.60.24.13 (executor 1) (1/3) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_123_piece0 in memory on 10.60.24.10:41041 (size: 17.5 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO TaskSetManager: Finished task 2.0 in stage 187.0 (TID 1371) in 18 ms on 10.60.24.10 (executor 0) (2/3) 23/06/28 07:51:27 INFO TaskSetManager: Finished task 0.0 in stage 187.0 (TID 1369) in 41 ms on 10.60.24.10 (executor 0) (3/3) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Removed TaskSet 187.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:27 INFO DAGScheduler: ShuffleMapStage 187 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.049 s 23/06/28 07:51:27 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:27 INFO DAGScheduler: running: Set() 23/06/28 07:51:27 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:27 INFO DAGScheduler: failed: Set() 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:27 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":89197,"lifetime":385959},"10.60.24.13":{"diskUsage":74852,"lifetime":386229}}, ioCacheNumScanTasks={"numLocalScanTasks":33,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:51:27 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:51:27 INFO DAGScheduler: Got job 78 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:51:27 INFO DAGScheduler: Final stage: ResultStage 189 (collect at Checksum.scala:561) 23/06/28 07:51:27 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 188) 23/06/28 07:51:27 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:27 INFO DAGScheduler: Submitting ResultStage 189 (MapPartitionsRDD[399] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:51:27 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:27 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:27 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:27 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 189 (MapPartitionsRDD[399] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Adding task set 189.0 with 1 tasks resource profile 0 23/06/28 07:51:27 INFO FairSchedulableBuilder: Added task set TaskSet_189.0 tasks to pool 974573833245257412 23/06/28 07:51:27 INFO TaskSetManager: Starting task 0.0 in stage 189.0 (TID 1372) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_125 stored as values in memory (estimated size 502.8 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_125_piece0 stored as bytes in memory (estimated size 126.8 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_125_piece0 in memory on 10.60.24.15:40507 (size: 126.8 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 125 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_125_piece0 in memory on 10.60.24.10:41041 (size: 126.8 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 46 to 10.60.24.10:44288 23/06/28 07:51:27 INFO BlockManagerInfo: Added rdd_395_0 in memory on 10.60.24.10:41041 (size: 1421.0 B, free: 487.7 GiB) 23/06/28 07:51:27 INFO TaskSetManager: Finished task 0.0 in stage 189.0 (TID 1372) in 60 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Removed TaskSet 189.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:27 INFO DAGScheduler: ResultStage 189 (collect at Checksum.scala:561) finished in 0.080 s 23/06/28 07:51:27 INFO DAGScheduler: Job 78 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:27 INFO TaskSchedulerImpl: Killing all running tasks in stage 189: Stage finished 23/06/28 07:51:27 INFO DAGScheduler: Job 78 finished: collect at Checksum.scala:561, took 0.082093 s 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:51:27 INFO DAGScheduler: Registering RDD 402 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) as input to shuffle 47 23/06/28 07:51:27 INFO DAGScheduler: Got map stage job 79 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) with 1 output partitions 23/06/28 07:51:27 INFO DAGScheduler: Final stage: ShuffleMapStage 191 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) 23/06/28 07:51:27 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 190) 23/06/28 07:51:27 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:27 INFO DAGScheduler: Submitting ShuffleMapStage 191 (MapPartitionsRDD[402] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173), which has no missing parents 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:27 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:27 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:27 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:27 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 191 (MapPartitionsRDD[402] at $anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Adding task set 191.0 with 1 tasks resource profile 0 23/06/28 07:51:27 INFO FairSchedulableBuilder: Added task set TaskSet_191.0 tasks to pool 974573833245257412 23/06/28 07:51:27 INFO TaskSetManager: Starting task 0.0 in stage 191.0 (TID 1373) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_126 stored as values in memory (estimated size 473.2 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_126_piece0 stored as bytes in memory (estimated size 120.9 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_126_piece0 in memory on 10.60.24.15:40507 (size: 120.9 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 126 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_126_piece0 in memory on 10.60.24.10:41041 (size: 120.9 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO TaskSetManager: Finished task 0.0 in stage 191.0 (TID 1373) in 34 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Removed TaskSet 191.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:27 INFO DAGScheduler: ShuffleMapStage 191 ($anonfun$withThreadLocalCaptured$2 at OptimisticTransaction.scala:173) finished in 0.044 s 23/06/28 07:51:27 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:51:27 INFO DAGScheduler: running: Set() 23/06/28 07:51:27 INFO DAGScheduler: waiting: Set() 23/06/28 07:51:27 INFO DAGScheduler: failed: Set() 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:51:27 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:51:27 INFO DAGScheduler: Got job 80 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:51:27 INFO DAGScheduler: Final stage: ResultStage 194 (first at Snapshot.scala:271) 23/06/28 07:51:27 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 193) 23/06/28 07:51:27 INFO DAGScheduler: Missing parents: List() 23/06/28 07:51:27 INFO DAGScheduler: Submitting ResultStage 194 (MapPartitionsRDD[404] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:51:27 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:51:27 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:51:27 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:51:27 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 194 (MapPartitionsRDD[404] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Adding task set 194.0 with 1 tasks resource profile 0 23/06/28 07:51:27 INFO FairSchedulableBuilder: Added task set TaskSet_194.0 tasks to pool 974573833245257412 23/06/28 07:51:27 INFO TaskSetManager: Starting task 0.0 in stage 194.0 (TID 1374) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_127 stored as values in memory (estimated size 419.4 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO MemoryStore: Block broadcast_127_piece0 stored as bytes in memory (estimated size 110.4 KiB, free 750.6 GiB) 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_127_piece0 in memory on 10.60.24.15:40507 (size: 110.4 KiB, free: 750.7 GiB) 23/06/28 07:51:27 INFO SparkContext: Created broadcast 127 from broadcast at TaskSetManager.scala:627 23/06/28 07:51:27 INFO BlockManagerInfo: Added broadcast_127_piece0 in memory on 10.60.24.13:44035 (size: 110.4 KiB, free: 487.7 GiB) 23/06/28 07:51:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 47 to 10.60.24.13:58548 23/06/28 07:51:27 INFO TaskSetManager: Finished task 0.0 in stage 194.0 (TID 1374) in 39 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:51:27 INFO TaskSchedulerImpl: Removed TaskSet 194.0, whose tasks have all completed, from pool 974573833245257412 23/06/28 07:51:27 INFO DAGScheduler: ResultStage 194 (first at Snapshot.scala:271) finished in 0.053 s 23/06/28 07:51:27 INFO DAGScheduler: Job 80 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:51:27 INFO TaskSchedulerImpl: Killing all running tasks in stage 194: Stage finished 23/06/28 07:51:27 INFO DAGScheduler: Job 80 finished: first at Snapshot.scala:271, took 0.054354 s 23/06/28 07:51:27 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:51:27 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log, version=62, metadata=Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)), logSegment=LogSegment(dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log,62,ArrayBuffer(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000061.json; isDirectory=false; length=1943; replication=1; blocksize=268435456; modification_time=1687938684000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC785C04B5"'}, VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.json; isDirectory=false; length=1364; replication=1; blocksize=268435456; modification_time=1687938687000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=root; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8DB77AC7A258EF7"'}),WrappedArray(VersionedFileStatus{VersionedFileStatus{path=dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000060.checkpoint.parquet; isDirectory=false; length=29625; replication=1; blocksize=268435456; modification_time=1687937289000; access_time=0; owner=7a0a3363-6b33-46a2-983f-dac0367dae1b; group=$superuser; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='0x8DB77A938E70CD4'}),Some(60),1687938687000), checksumOpt=Some(VersionChecksum(Some(ae7c3139-06d4-4bc6-a1df-05c133e75a24),86107,6,None,None,1,1,Some(Stream()),Metadata(5d0ae5b6-42a3-47d1-8d94-be2429376ace,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},List(),Map(),Some(1642649700153)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@558ef978,[J@68ffc89c)),None,Some(Stream(AddFile(part-00000-6b9b3a1d-a8da-45de-a4b3-8cd50f6ca176-c000.snappy.parquet,Map(),1392,1658882351000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-0321307b-64b5-4330-9ea7-f4a1546c9fc7-c000.snappy.parquet,Map(),1392,1658899177000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-78d99c18-4585-493e-80ff-883714b186ca-c000.snappy.parquet,Map(),1392,1642649707000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-47a3c79f-1f53-4b6a-80c4-bd71f2f757a8-c000.snappy.parquet,Map(),79147,1658899170000,false,{"numRecords":5752,"minValues":{"component":"","groupid":3,"id":2,"itemid":0,"timeadded":1529302926,"userid":0,"_DLRawZoneTimeStamp":"2022-01-20T14:30:28.955Z","_DLTrustedZoneTimeStamp":"2022-01-20T14:35:20.000Z","_RecordStart":"2022-01-20T14:35:20.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":0,"_RecordCurrent":1},"maxValues":{"component":"","groupid":9686,"id":9907,"itemid":0,"timeadded":1653010486,"userid":2376,"_DLRawZoneTimeStamp":"2022-07-26T18:00:15.455Z","_DLTrustedZoneTimeStamp":"2022-07-26T18:11:19.000Z","_RecordStart":"2022-07-26T18:10:50.000Z","_RecordEnd":"9999-12-31T00:00:00.000Z","_RecordDeleted":1,"_RecordCurrent":1},"nullCount":{"component":0,"groupid":0,"id":0,"itemid":0,"timeadded":0,"userid":0,"_DLRawZoneTimeStamp":0,"_DLTrustedZoneTimeStamp":0,"_RecordStart":0,"_RecordEnd":0,"_RecordDeleted":0,"_RecordCurrent":0}},null,null), AddFile(part-00000-19db40fb-349f-41e5-af51-2ddf2517f348-c000.snappy.parquet,Map(),1392,1643247051000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null), AddFile(part-00000-b3bce997-2d4c-43b0-94ba-039b842395a3-c000.snappy.parquet,Map(),1392,1642649740000,false,{"numRecords":0,"minValues":{},"maxValues":{},"nullCount":{}},null,null)))))) 23/06/28 07:51:27 INFO MapPartitionsRDD: Removing RDD 323 from persistence list 23/06/28 07:51:27 INFO BlockManager: Removing RDD 323 23/06/28 07:51:27 INFO OptimisticTransaction: [tableId=5d0ae5b6,txnId=ae7c3139] Committed delta #62 to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log 23/06/28 07:51:27 INFO ChecksumHook: Writing checksum file for table path dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log version 62 23/06/28 07:51:27 INFO AzureBlobFileSystem:V3: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:51:27 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.crc using temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp 23/06/28 07:51:27 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp] Closing stream; size: 0 23/06/28 07:51:27 INFO AbfsOutputStream: FS_OP_CREATE FILE[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp] Upload complete; size: 5149 23/06/28 07:51:27 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.crc] Starting rename. Issuing rename operation. 23/06/28 07:51:27 INFO AzureBlobFileSystem:V3: FS_OP_RENAME SRC[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp] DST[abfss://trusted@zedlsbi01t.dfs.core.windows.net/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.crc] Rename successful. 23/06/28 07:51:27 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/__tmp_path_dir/.00000000000000000062.crc.3bb87673-88d5-40ad-a99b-d4a935486f18.tmp to dbfs:/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta/_delta_log/00000000000000000062.crc 23/06/28 07:51:27 INFO MapPartitionsRDD: Removing RDD 356 from persistence list 23/06/28 07:51:27 INFO ClusterLoadMonitor: Removed query with execution ID:178. Current active queries:0 23/06/28 07:51:27 INFO BlockManager: Removing RDD 356 23/06/28 07:51:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:51:27 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:51:27.867Z","run":{"runId":"3538b3ba-923e-4a7d-8001-248d6c9fe335","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.MergeIntoCommandEdge","num-children":0,"source":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"STG","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"merge_key","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.And","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Join","num-children":2,"left":0,"right":1,"joinType":{"object":"org.apache.spark.sql.catalyst.plans.LeftOuter$"},"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[SRC]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.Nvl","num-children":1,"left":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"}],"right":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"replacement":0},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[TGT]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"}],"hint":null},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"TGT","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4760,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4761,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4762,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4763,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4764,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"SRC","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, raw]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4775,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4776,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4777,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4778,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4779,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4780,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4781,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"year","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4782,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"month","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4783,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"day","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4784,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"raw","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"year","type":"string","nullable":true,"metadata":{}},{"name":"month","type":"string","nullable":true,"metadata":{}},{"name":"day","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649443000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":false,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"target":[{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"MN","qualifier":[]},"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4741,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4742,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4743,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4744,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4745,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4746,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4747,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4748,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4749,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4750,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4751,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"targetFileIndex":null,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.Coalesce","num-children":2,"children":[0,1]},{"class":"org.apache.spark.sql.catalyst.expressions.Cast","num-children":1,"child":0,"dataType":"string","timeZoneId":"Etc/UTC","evalMode":null},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"ID","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4743,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"merge_key","dataType":"string","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4728,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"}],"matchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoMatchedUpdateClause","num-children":13,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}],"actions":[1,2,3,4,5,6,7,8,9,10,11,12]},{"class":"org.apache.spark.sql.catalyst.expressions.EqualTo","num-children":2,"left":0,"right":1},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[MN]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4749,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4750,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4752,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"notMatchedClauses":[[{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeIntoNotMatchedInsertClause","num-children":12,"actions":[0,1,2,3,4,5,6,7,8,9,10,11]},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[component]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4753,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[groupid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4754,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[id]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4755,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[itemid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4756,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[timeadded]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4757,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[userid]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4758,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLRawZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":4759,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[STG]"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_DLTrustedZoneTimeStamp]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordStart]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"2023-06-28 17:51:25","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordEnd]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"9999-12-31 00:00:00","dataType":"timestamp"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordDeleted]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeltaMergeAction","num-children":1,"targetColNameParts":"[_RecordCurrent]","expr":0,"targetColNameResolved":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"1","dataType":"integer"}]],"notMatchedBySourceClauses":[],"migratedSchema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"trackHighWaterMarks":{"object":"scala.collection.immutable.Set$EmptySet$"}}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_merge_into_command_edge","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:51:27 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_5011235230273436863_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:27 INFO ProgressReporter$: Removed result fetcher for 7716632506684841697_4914625121334659537_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:27 INFO ProgressReporter$: Added result fetcher for 974573833245257412_4914625121334659537_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:28 INFO ProgressReporter$: Removed result fetcher for 974573833245257412_4914625121334659537_job-409724359447930-run-5532185-action-4028745939396423 23/06/28 07:51:28 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:51:31 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:51:32 INFO DriverCorral$: Cleaning the wrapper ReplId-6b16f-c615f-2e9ee-1 (currently in status Idle(ReplId-6b16f-c615f-2e9ee-1)) 23/06/28 07:51:32 INFO DriverCorral$: sending shutdown signal for REPL ReplId-6b16f-c615f-2e9ee-1 23/06/28 07:51:32 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-6b16f-c615f-2e9ee-1 23/06/28 07:51:32 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-6b16f-c615f-2e9ee-1 23/06/28 07:51:32 INFO DriverCorral$: ReplId-6b16f-c615f-2e9ee-1 successfully discarded 23/06/28 07:51:32 INFO DriverCorral$: Cleaning the wrapper ReplId-d8661-bce1f-96ec4 (currently in status Idle(ReplId-d8661-bce1f-96ec4)) 23/06/28 07:51:32 INFO DriverCorral$: sending shutdown signal for REPL ReplId-d8661-bce1f-96ec4 23/06/28 07:51:32 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:51:32 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:51:32 INFO ReplCrashUtils$: python shell exit code: 143 23/06/28 07:51:32 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-d8661-bce1f-96ec4 23/06/28 07:51:32 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-d8661-bce1f-96ec4 23/06/28 07:51:32 INFO DriverCorral$: ReplId-d8661-bce1f-96ec4 successfully discarded 23/06/28 07:52:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:52:04 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:52:04 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:52:04 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:52:04 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:52:04 INFO DriverCorral: Starting python repl ReplId-5e6c8-2fbdc-d9b49-5 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:188. Current active queries:1 23/06/28 07:52:04 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$ 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:188. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:189. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:189. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:190. Current active queries:1 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:190. Current active queries:0 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:04.027Z","run":{"runId":"ba3ff42f-2bbe-4b92-84ce-24b7110e1ad2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:191. Current active queries:1 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.027Z","run":{"runId":"ba3ff42f-2bbe-4b92-84ce-24b7110e1ad2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:191. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:192. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:192. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:193. Current active queries:1 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.032Z","run":{"runId":"370a8823-9930-4b2c-873c-a41bd1274706","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:193. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:194. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:194. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:195. Current active queries:1 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.036Z","run":{"runId":"91450dc1-c5fb-4aed-8529-8980a9d2f5ae","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:195. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:196. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:196. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:197. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:197. Current active queries:0 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.04Z","run":{"runId":"a24a67cf-12fd-469d-bf97-10ebc6d31d04","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:198. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:198. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:199. Current active queries:1 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:199. Current active queries:0 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.044Z","run":{"runId":"633db2e5-6c8c-4a5f-acb5-f1977cc0a6b3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:200. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:200. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:201. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:201. Current active queries:0 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.048Z","run":{"runId":"1bbbac2d-761c-4697-a0b8-a9eaa9d36f75","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:202. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:202. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:203. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:203. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.051Z","run":{"runId":"1789ca37-ece6-4b8c-aa09-632744febe4f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:204. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:204. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:205. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:205. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:206. Current active queries:1 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.055Z","run":{"runId":"9afdb39c-b587-41fc-a817-ab113e9f3aa7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:206. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:207. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:207. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.059Z","run":{"runId":"d6d6a9e1-af9d-43bd-a0d7-859d2831c41d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:208. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:208. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:209. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:209. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.062Z","run":{"runId":"98845559-f221-478b-8cad-1ae09d5f15fe","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:210. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:210. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:211. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:211. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.067Z","run":{"runId":"7cb11f93-f5d5-4b38-85da-959bc84e70aa","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:212. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:212. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:213. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:213. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.07Z","run":{"runId":"17c85cf9-4fe8-4ae3-86d6-f85d8bdcddf7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:214. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:214. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:215. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:215. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.074Z","run":{"runId":"2acd9a6b-e27f-47d9-a336-113ab21a17f6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:216. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:216. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:217. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:217. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:218. Current active queries:1 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.078Z","run":{"runId":"ab014fbc-0278-475d-bea9-73a5ff79ffcc","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:218. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:219. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:219. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:220. Current active queries:1 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.082Z","run":{"runId":"2702055a-aad0-4e2b-b7d2-4c51bedf17b7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:220. Current active queries:0 23/06/28 07:52:04 INFO ClusterLoadMonitor: Added query with execution ID:221. Current active queries:1 23/06/28 07:52:04 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:04 INFO ClusterLoadMonitor: Removed query with execution ID:221. Current active queries:0 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.085Z","run":{"runId":"ae4b11aa-34bd-4e4d-9c02-e0d56afdcdcb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO JupyterDriverLocal: Starting gateway server for repl ReplId-5e6c8-2fbdc-d9b49-5 23/06/28 07:52:04 INFO PythonPy4JUtil: Using pinned thread mode in Py4J 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.089Z","run":{"runId":"0aafde9b-72e7-4bd1-89fa-e83530773885","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.093Z","run":{"runId":"0a4fa8ce-034a-4975-badf-616fea4a2ae1","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.096Z","run":{"runId":"76034df6-99d6-42d1-ab62-e3369aa23468","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.1Z","run":{"runId":"18c2fb27-f07e-4198-956a-359d41afea50","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.104Z","run":{"runId":"f4dafb69-7585-43c2-b05b-a969fbf2425a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.108Z","run":{"runId":"ea2cb299-5daa-477a-96e9-fb986369354c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.112Z","run":{"runId":"cadf5a4f-f6cc-47a8-a55d-5e45858bebc6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.116Z","run":{"runId":"881146ac-80cc-41f6-98ef-6944dae84485","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.12Z","run":{"runId":"dfed809c-dca6-4395-abc0-006000e0e7cf","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.123Z","run":{"runId":"6ed92ac4-2785-42d0-ab11-9d1b2edc5633","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.127Z","run":{"runId":"ad2b5934-4596-43a1-9261-bc000f89321c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.131Z","run":{"runId":"02ff1867-f285-4ed1-8f6e-a3c3ffb4a519","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.134Z","run":{"runId":"b57cb380-ea39-423f-9dc3-02ca95698f1c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.138Z","run":{"runId":"ec4f5b4d-a583-4d07-a4df-9443fc3bae8a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.141Z","run":{"runId":"7e5b4d5e-9b39-4d3b-bb5d-c83066796b65","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.145Z","run":{"runId":"e44783ea-b8f9-46e8-a6b4-fa5a802ab594","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.149Z","run":{"runId":"cc885d6b-c911-4413-b123-860c5bf0a2aa","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:04 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:04.152Z","run":{"runId":"0ce1305a-4fd1-4f0f-9b27-dd69ae87cf26","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:04 INFO VirtualenvCloneHelper: Creating notebook-scoped virtualenv for 9d7d944e-7f3e-418c-827d-796373002124 23/06/28 07:52:04 INFO Utils: resolved command to be run: List(virtualenv, /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124, -p, /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, --no-download, --no-setuptools, --no-wheel) 23/06/28 07:52:04 INFO PythonEnvCloneHelper$: Created python virtualenv: /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124 23/06/28 07:52:04 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, -c, import sys; dirs=[p for p in sys.path if 'package' in p]; print(' '.join(dirs))) 23/06/28 07:52:04 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124/bin/python, -c, from distutils.sysconfig import get_python_lib; print(get_python_lib())) 23/06/28 07:52:04 INFO PythonEnvCloneHelper$: Created sites.pth at /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124/lib/python3.9/site-packages/sites.pth 23/06/28 07:52:04 INFO NotebookScopedPythonEnvManager: Time spent to start virtualenv /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124 is 213(ms) 23/06/28 07:52:04 INFO NotebookScopedPythonEnvManager: Registered /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124/lib/python3.9/site-packages with the WatchService sun.nio.fs.LinuxWatchService$LinuxWatchKey@1cd95d57 23/06/28 07:52:04 INFO IpykernelUtils$: Python process builder: [bash, /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124/python_start_9d7d944e-7f3e-418c-827d-796373002124.sh, /databricks/spark/python/pyspark/wrapped_python.py, root, /local_disk0/.ephemeral_nfs/envs/pythonEnv-9d7d944e-7f3e-418c-827d-796373002124/bin/python, /databricks/python_shell/scripts/db_ipykernel_launcher.py, -f, /databricks/kernel-connections/7b8241270d081b04d8a0d78bbd7ec803313c89d987fc24271d1acc8d997b1702.json] 23/06/28 07:52:04 INFO IpykernelUtils$: Cgroup isolation disabled, not placing python process in repl cgroup 23/06/28 07:52:06 INFO PythonDriverWrapper: setupRepl:ReplId-5e6c8-2fbdc-d9b49-5: finished to load 23/06/28 07:52:06 INFO DriverCorral: Starting scala repl ReplId-54980-b86c0-994c6-a 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:222. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:222. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:223. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:223. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:224. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:06.103Z","run":{"runId":"a2ba3f48-cf96-450d-a976-0d73b137743e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:224. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:225. Current active queries:1 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.104Z","run":{"runId":"a2ba3f48-cf96-450d-a976-0d73b137743e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:225. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:226. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:226. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:227. Current active queries:1 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.109Z","run":{"runId":"5689b987-6b64-40a0-ac57-d62d87ebba56","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:227. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:228. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:228. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:229. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.113Z","run":{"runId":"396307ce-b7f5-4179-b332-e909fa10fc31","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:229. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:230. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:230. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:231. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.116Z","run":{"runId":"43bca6c9-8679-48d0-90a7-a3428336be6a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:231. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:232. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:232. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:233. Current active queries:1 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.12Z","run":{"runId":"9951bd22-947f-40ee-b8de-10644203d996","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:233. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:234. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:234. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.124Z","run":{"runId":"65243912-4b9d-442f-a788-063afa9c2ca2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:235. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:235. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:236. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:236. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.128Z","run":{"runId":"b4df86a1-de0a-4111-ba97-d6f6e9299c20","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:237. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:237. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:238. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:238. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.132Z","run":{"runId":"0bf8e7a3-80a1-4932-92cf-8293a9c1c017","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:239. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:239. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:240. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:240. Current active queries:0 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.136Z","run":{"runId":"972f66b5-da8c-4622-bdf1-a815330dccc0","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:241. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:241. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:242. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.139Z","run":{"runId":"5690d249-cf7d-4ea3-9b5a-ec1ce03af8ec","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:242. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:243. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:243. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:244. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.144Z","run":{"runId":"5aa3d386-6735-4a48-a4e5-f721a4efd984","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:244. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:245. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:245. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:246. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.147Z","run":{"runId":"d166ea96-18bb-4108-a621-b68a09026392","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:246. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:247. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:247. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:248. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.151Z","run":{"runId":"8b38e828-e57d-49e7-ac70-9ae572bc6d7a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:248. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:249. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:249. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:250. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.155Z","run":{"runId":"ed841abb-6a7c-4fca-84ca-ba75e0b72b7f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:250. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:251. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:251. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:252. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.159Z","run":{"runId":"96a4df62-ad17-4383-85d1-0aa018aa1b93","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:252. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:253. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:253. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:254. Current active queries:1 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.163Z","run":{"runId":"21ed97df-c48b-4eaa-8cb8-8a94d873e755","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:254. Current active queries:0 23/06/28 07:52:06 INFO ClusterLoadMonitor: Added query with execution ID:255. Current active queries:1 23/06/28 07:52:06 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:06 INFO ClusterLoadMonitor: Removed query with execution ID:255. Current active queries:0 23/06/28 07:52:06 INFO ScalaDriverWrapper: setupRepl:ReplId-54980-b86c0-994c6-a: finished to load 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.166Z","run":{"runId":"f4cea931-b81b-4ad1-a678-4fdb750866b6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5708471040524947107_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.17Z","run":{"runId":"8d6e1bc0-d122-469d-a7ea-51aa66af07a5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5708471040524947107_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.174Z","run":{"runId":"ce224bb8-558a-4dae-87ff-ae27668da77b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.178Z","run":{"runId":"37231474-d2f4-4a8e-9262-080ad04b3076","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.183Z","run":{"runId":"ac26a55d-72e2-4d2b-ad7b-307c31340aeb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.187Z","run":{"runId":"1ced517f-5b76-44a7-b6c1-ee1f2782d7c9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.191Z","run":{"runId":"dfbcb14d-c8c5-4bad-b0c3-8bff1bda0d56","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5708471040524947107_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.195Z","run":{"runId":"de2649c3-82a3-4d78-ba22-065defd674a7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5601238732833943506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5601238732833943506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.198Z","run":{"runId":"751799e5-4067-4f70-8e1f-fd6259ef8fbb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.202Z","run":{"runId":"37c705c1-fa00-492e-b414-0d9c73073ac9","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.206Z","run":{"runId":"339b413c-257c-44f1-a1a5-cc70dc9f3c84","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.21Z","run":{"runId":"1612ac68-79f1-4eb4-8b37-ba09e625e381","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.214Z","run":{"runId":"e561bafe-62d9-4d1f-8ea6-4b90dae32ab4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.217Z","run":{"runId":"4406c28d-6409-4ba7-a835-f13e834b6060","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.221Z","run":{"runId":"64b30248-4ee1-4496-bcbd-2f887ced9ddd","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.225Z","run":{"runId":"9493682a-0fa2-4e53-a907-c77fb0ee616b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.229Z","run":{"runId":"bc18af2d-d229-4cc1-9ebe-c23cb9d34b95","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:06 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:06.233Z","run":{"runId":"e5bd42b4-1e01-4383-9d77-11f1b50454f7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5601238732833943506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6732135454053990757_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6732135454053990757_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6732135454053990757_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7935442797766534262_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7935442797766534262_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7935442797766534262_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7266064368056242596_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7266064368056242596_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7266064368056242596_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5032967530319329647_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5032967530319329647_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInRead' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInRead' instead. 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5032967530319329647_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7164798507867287860_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7164798507867287860_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInWrite' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInWrite' instead. 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7164798507867287860_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5312989986708822662_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5312989986708822662_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5312989986708822662_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:06 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7587036594590804009_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:07 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7587036594590804009_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:07 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7587036594590804009_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:07 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6470767931681208500_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:07 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6470767931681208500_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:07 INFO DriverILoop: Set class prefix to: $lineb8001a5da773407784f18637eb9008b9 23/06/28 07:52:07 INFO DriverILoop: set ContextClassLoader 23/06/28 07:52:07 INFO DriverILoop: initialized intp 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6470767931681208500_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7239419857199102683_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7239419857199102683_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7239419857199102683_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5838086133266447361_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5838086133266447361_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5838086133266447361_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4988901241312015415_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4988901241312015415_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4988901241312015415_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5588144353815346458_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5588144353815346458_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5588144353815346458_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8358878048093000531_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8358878048093000531_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8358878048093000531_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7784588058959557217_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7784588058959557217_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7784588058959557217_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5676393138802992964_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5676393138802992964_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:09 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5676393138802992964_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6595126293926510707_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6595126293926510707_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6595126293926510707_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7507452312587223456_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7507452312587223456_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7507452312587223456_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6896638375936392423_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6896638375936392423_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6896638375936392423_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4848712095855166133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4848712095855166133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4848712095855166133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9149213660133396914_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9149213660133396914_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9149213660133396914_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6253185558261082087_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6253185558261082087_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6253185558261082087_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8165276031680215371_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8165276031680215371_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8165276031680215371_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8536083818802597099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8536083818802597099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8536083818802597099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8756414117621066507_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8756414117621066507_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8756414117621066507_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4613335296624487250_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:10 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4613335296624487250_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4613335296624487250_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5275996728547333254_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5275996728547333254_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5275996728547333254_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6222594541845904824_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6222594541845904824_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6222594541845904824_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7297554437372153938_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7297554437372153938_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7297554437372153938_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7392958744441379576_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7392958744441379576_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7392958744441379576_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5242268731819936274_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5242268731819936274_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5242268731819936274_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7284615119761100238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7284615119761100238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7284615119761100238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5582686943019185321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5582686943019185321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5582686943019185321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5890076782853375454_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5890076782853375454_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5890076782853375454_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8098593026555869836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8098593026555869836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8098593026555869836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7174118649135012735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:11 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7174118649135012735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7174118649135012735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7758715151034267689_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7758715151034267689_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7758715151034267689_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6597812705398375145_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6597812705398375145_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6597812705398375145_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7959437793314820939_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7959437793314820939_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7959437793314820939_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7981646805730078637_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7981646805730078637_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7981646805730078637_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7584311048625309131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7584311048625309131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7584311048625309131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5374606350589806903_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5374606350589806903_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5374606350589806903_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5905920898557210220_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5905920898557210220_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5905920898557210220_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4859800873353974352_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4859800873353974352_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4859800873353974352_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9159966746369048497_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9159966746369048497_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9159966746369048497_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5815613217146182569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:12 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5815613217146182569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5815613217146182569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4704813523667199255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4704813523667199255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4704813523667199255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6987635010604999099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6987635010604999099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6987635010604999099_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5375566540512141501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5375566540512141501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5375566540512141501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7481846377686440403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7481846377686440403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7481846377686440403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5293483487260878409_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5293483487260878409_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5293483487260878409_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5902733990109783465_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5902733990109783465_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5902733990109783465_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8654256519362077847_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8654256519362077847_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8654256519362077847_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4622842228438567169_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4622842228438567169_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4622842228438567169_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6727548960450101556_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6727548960450101556_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6727548960450101556_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6505288791788024639_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:13 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6505288791788024639_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6505288791788024639_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6151036380831238255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6151036380831238255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6151036380831238255_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6691684678839482188_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6691684678839482188_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6691684678839482188_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4938823689999302762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4938823689999302762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4938823689999302762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6387488646243324462_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6387488646243324462_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6387488646243324462_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6108748075345761735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6108748075345761735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6108748075345761735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6148905976167634817_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6148905976167634817_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6148905976167634817_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6815679304709396652_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6815679304709396652_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6815679304709396652_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6255343270458062501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6255343270458062501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6255343270458062501_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8522084866121334167_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8522084866121334167_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8522084866121334167_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7126781834301612061_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:14 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7126781834301612061_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7126781834301612061_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7907664859024906266_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7907664859024906266_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7907664859024906266_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7365691729393311267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7365691729393311267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7365691729393311267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6271537649941061238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6271537649941061238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6271537649941061238_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6854001008717023790_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6854001008717023790_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6854001008717023790_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6933521933381660452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6933521933381660452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6933521933381660452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8461654776914064701_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8461654776914064701_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8461654776914064701_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5256849545157711297_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5256849545157711297_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5256849545157711297_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7598592461768464306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7598592461768464306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7598592461768464306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5829993661716145873_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5829993661716145873_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5829993661716145873_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5585341892582269080_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:15 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5585341892582269080_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5585341892582269080_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6259710560323768727_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6259710560323768727_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6259710560323768727_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6522933184060478489_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6522933184060478489_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6522933184060478489_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8784607242626792648_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8784607242626792648_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8784607242626792648_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6437163751489235267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6437163751489235267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6437163751489235267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6990540741231176650_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6990540741231176650_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6990540741231176650_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8599230138403826321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8599230138403826321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8599230138403826321_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6425905447553522722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6425905447553522722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6425905447553522722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9169482348480352471_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9169482348480352471_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9169482348480352471_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4661907064926959976_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4661907064926959976_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4661907064926959976_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6172537795465920429_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:16 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6172537795465920429_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6172537795465920429_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6722898566891880191_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6722898566891880191_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6722898566891880191_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5120289376317028365_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5120289376317028365_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5120289376317028365_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5742063800358283506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5742063800358283506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5742063800358283506_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7431968827765826669_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7431968827765826669_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7431968827765826669_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7039183444709543603_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7039183444709543603_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7039183444709543603_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5210375510638422445_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5210375510638422445_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5210375510638422445_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5686365156990564929_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5686365156990564929_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5686365156990564929_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9103875312158807487_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9103875312158807487_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9103875312158807487_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4860114056514257236_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4860114056514257236_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4860114056514257236_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8607841121550705547_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:17 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8607841121550705547_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8607841121550705547_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6796610234815289601_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6796610234815289601_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6796610234815289601_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6479436979920469716_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6479436979920469716_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6479436979920469716_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6771473206610296633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6771473206610296633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6771473206610296633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8500595545064714044_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8500595545064714044_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8500595545064714044_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5974168855560253329_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5974168855560253329_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5974168855560253329_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6000525662317975722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6000525662317975722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6000525662317975722_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8962458022357772252_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8962458022357772252_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8962458022357772252_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7766441540539427128_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7766441540539427128_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7766441540539427128_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6315924729447740571_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6315924729447740571_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6315924729447740571_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5209817904336787899_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:18 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5209817904336787899_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5209817904336787899_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7615685483491208129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7615685483491208129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7615685483491208129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4810591713975539268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4810591713975539268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4810591713975539268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6328393615442918373_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6328393615442918373_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6328393615442918373_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6025968606165547591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6025968606165547591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6025968606165547591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5618054647754944122_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5618054647754944122_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5618054647754944122_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6057747003743086813_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6057747003743086813_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6057747003743086813_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6610757818258893750_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6610757818258893750_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6610757818258893750_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7343761089460903286_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7343761089460903286_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7343761089460903286_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8245084216868388612_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8245084216868388612_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8245084216868388612_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4882539459703919721_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:19 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4882539459703919721_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4882539459703919721_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9131067153904787602_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9131067153904787602_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9131067153904787602_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6448797877145878775_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6448797877145878775_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6448797877145878775_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6153892286943438439_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6153892286943438439_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6153892286943438439_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5994677607728170453_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5994677607728170453_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5994677607728170453_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7879827346248891855_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7879827346248891855_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7879827346248891855_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5976972068946754524_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5976972068946754524_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5976972068946754524_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7364628929579294486_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7364628929579294486_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7364628929579294486_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5949393250008011432_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5949393250008011432_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5949393250008011432_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8058053398589534309_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8058053398589534309_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8058053398589534309_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8074372398103897534_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:20 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8074372398103897534_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8074372398103897534_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7241148365992940237_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7241148365992940237_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7241148365992940237_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4983172738541622455_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4983172738541622455_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4983172738541622455_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7070509240770519380_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7070509240770519380_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7070509240770519380_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5714533327047985591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5714533327047985591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5714533327047985591_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5604784468677786734_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5604784468677786734_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5604784468677786734_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8897924113093665532_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8897924113093665532_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8897924113093665532_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7004209014959424171_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7004209014959424171_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7004209014959424171_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5825288194599731287_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5825288194599731287_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5825288194599731287_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6231098435010592839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6231098435010592839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6231098435010592839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4697878799809111917_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:21 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4697878799809111917_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4697878799809111917_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6028201118844228020_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6028201118844228020_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6028201118844228020_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7283035203176901398_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7283035203176901398_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7283035203176901398_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7972344137817885260_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7972344137817885260_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7972344137817885260_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8491183717100310719_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8491183717100310719_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8491183717100310719_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4901236414440182697_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4901236414440182697_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4901236414440182697_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5710019323819575086_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5710019323819575086_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5710019323819575086_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8129227885244190918_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8129227885244190918_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8129227885244190918_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7313657539681469055_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7313657539681469055_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7313657539681469055_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4833690814845430382_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4833690814845430382_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4833690814845430382_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5653926333247281826_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:22 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5653926333247281826_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5653926333247281826_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5263539851475852614_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5263539851475852614_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5263539851475852614_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6125853617110849801_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6125853617110849801_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6125853617110849801_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8163551578337632585_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8163551578337632585_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8163551578337632585_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6544121907461525047_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6544121907461525047_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6544121907461525047_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5054379000931887543_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5054379000931887543_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5054379000931887543_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6463335159309210268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6463335159309210268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6463335159309210268_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4888649391517735417_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4888649391517735417_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4888649391517735417_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5866553242908091667_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5866553242908091667_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5866553242908091667_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7276909531529175289_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7276909531529175289_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7276909531529175289_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8946186192826541354_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:23 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8946186192826541354_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8946186192826541354_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5177030339078480808_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5177030339078480808_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5177030339078480808_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6530270100388894708_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6530270100388894708_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6530270100388894708_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7574337360713947441_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7574337360713947441_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7574337360713947441_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5158830504044083176_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5158830504044083176_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5158830504044083176_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7584195720539717569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7584195720539717569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7584195720539717569_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6620727547878546447_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6620727547878546447_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6620727547878546447_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4802917941051434671_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4802917941051434671_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4802917941051434671_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7556770950518626464_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7556770950518626464_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7556770950518626464_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7180313030903807097_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7180313030903807097_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7180313030903807097_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7746248300207007392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:24 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7746248300207007392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7746248300207007392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6142279106582136747_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6142279106582136747_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6142279106582136747_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6222502587720188798_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6222502587720188798_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6222502587720188798_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6786482843191185675_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6786482843191185675_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6786482843191185675_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7797494026069050090_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7797494026069050090_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7797494026069050090_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8643016756793127518_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8643016756793127518_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8643016756793127518_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4629671293057647694_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_4629671293057647694_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4629671293057647694_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6246534854380922787_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6246534854380922787_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6246534854380922787_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5597467252140783222_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_5597467252140783222_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5597467252140783222_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8220552928784484516_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8220552928784484516_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8220552928784484516_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4641981211940859152_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:25 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_4641981211940859152_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4641981211940859152_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7138300324243168515_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7138300324243168515_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7138300324243168515_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6493076848725826783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6493076848725826783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6493076848725826783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9157250714771405033_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_9157250714771405033_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9157250714771405033_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5761117044430959094_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_5761117044430959094_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5761117044430959094_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7475895846027738017_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7475895846027738017_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7475895846027738017_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6572714387129863488_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6572714387129863488_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6572714387129863488_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8006306354304017582_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8006306354304017582_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8006306354304017582_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7711956941899984908_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7711956941899984908_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7711956941899984908_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5267791730064532131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_5267791730064532131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5267791730064532131_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7075228804917860105_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7075228804917860105_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7075228804917860105_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9106036196888041893_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:26 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_9106036196888041893_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9106036196888041893_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8315836149844184579_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8315836149844184579_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8315836149844184579_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7249060511321800836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_7249060511321800836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7249060511321800836_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4994191942178565302_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_4994191942178565302_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4994191942178565302_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4925885798252725348_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_4925885798252725348_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4925885798252725348_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8096950772381954699_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_8096950772381954699_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8096950772381954699_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6807262876956886875_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6807262876956886875_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6807262876956886875_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6893594179232688592_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_6893594179232688592_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6893594179232688592_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8862097768619556249_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8862097768619556249_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8862097768619556249_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5424571857918620403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:27 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5424571857918620403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5424571857918620403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9211478090259305084_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9211478090259305084_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9211478090259305084_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5378524321117653259_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5378524321117653259_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5378524321117653259_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5710754043583961952_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5710754043583961952_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5710754043583961952_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5391209011631378859_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5391209011631378859_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5391209011631378859_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9078355613512469098_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9078355613512469098_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9078355613512469098_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8772112823950067978_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8772112823950067978_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8772112823950067978_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6863963184783585172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6863963184783585172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6863963184783585172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5913665820350200341_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5913665820350200341_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5913665820350200341_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6352309939274719110_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6352309939274719110_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6352309939274719110_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6065172939927253379_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:28 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6065172939927253379_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6065172939927253379_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8025760901837601248_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8025760901837601248_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8025760901837601248_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7756588392156958403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7756588392156958403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7756588392156958403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8371372823875966789_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8371372823875966789_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8371372823875966789_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6417188919573684133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6417188919573684133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6417188919573684133_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7973004386466915403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7973004386466915403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7973004386466915403_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5651939897698916129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5651939897698916129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5651939897698916129_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8312192296705011392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8312192296705011392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8312192296705011392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8593879639633234022_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8593879639633234022_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8593879639633234022_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7969892324376672267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7969892324376672267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7969892324376672267_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8910709365395861953_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:29 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8910709365395861953_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8910709365395861953_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6143359913065170394_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6143359913065170394_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6143359913065170394_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8633412935289439762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8633412935289439762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8633412935289439762_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6846282672666811710_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6846282672666811710_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6846282672666811710_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7991094530235409806_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7991094530235409806_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7991094530235409806_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5837007671319219856_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5837007671319219856_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5837007671319219856_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6002962904772490509_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6002962904772490509_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6002962904772490509_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5421217572298059130_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5421217572298059130_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5421217572298059130_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5380973720570684932_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5380973720570684932_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5380973720570684932_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7125776343633703610_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7125776343633703610_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7125776343633703610_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7293504073280889160_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:30 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7293504073280889160_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7293504073280889160_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4630626834002139681_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4630626834002139681_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4630626834002139681_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5480237864220229324_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6095634769030761578_5480237864220229324_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5480237864220229324_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5060191390796557141_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5060191390796557141_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5060191390796557141_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7567331329883532364_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7567331329883532364_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7567331329883532364_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6009269761128348659_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6009269761128348659_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6009269761128348659_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7508341837719668296_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7508341837719668296_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7508341837719668296_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7667237607369276210_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7667237607369276210_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7667237607369276210_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5761284011816583243_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5761284011816583243_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5761284011816583243_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6028050819955460796_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6028050819955460796_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:31 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6028050819955460796_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_9176225741465951783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_9176225741465951783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_9176225741465951783_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4825805465817512926_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4825805465817512926_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4825805465817512926_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6915873291991793318_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6915873291991793318_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6915873291991793318_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6890864307015239177_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6890864307015239177_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6890864307015239177_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7740503247516754264_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7740503247516754264_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7740503247516754264_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8301029443640962633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8301029443640962633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8301029443640962633_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5043690040479657839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5043690040479657839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5043690040479657839_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5841952936820990452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5841952936820990452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5841952936820990452_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8746661027946659651_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8746661027946659651_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8746661027946659651_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8854154158371008874_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:32 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8854154158371008874_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8854154158371008874_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_4714091029672371564_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_4714091029672371564_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_4714091029672371564_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6188819656312247053_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6188819656312247053_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6188819656312247053_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8487777556210412385_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8487777556210412385_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8487777556210412385_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5084563699307386306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5084563699307386306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5084563699307386306_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8726943978782140298_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8726943978782140298_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8726943978782140298_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6416469984929143040_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6416469984929143040_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6416469984929143040_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5241116120273266313_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5241116120273266313_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5241116120273266313_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8305915093922182955_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8305915093922182955_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8305915093922182955_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7476778848852371615_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7476778848852371615_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7476778848852371615_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5891672885258492735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:33 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5891672885258492735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5891672885258492735_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5760094564387674162_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5760094564387674162_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5760094564387674162_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8450102456255900172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8450102456255900172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8450102456255900172_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8936850116558626056_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8936850116558626056_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8936850116558626056_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_8347743875614166434_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_8347743875614166434_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_8347743875614166434_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5790316445583497897_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5790316445583497897_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5790316445583497897_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6028773521036202030_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6028773521036202030_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6028773521036202030_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7767562413916327627_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7767562413916327627_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7767562413916327627_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6239315518638762014_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6239315518638762014_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6239315518638762014_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6265994647772692993_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6265994647772692993_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6265994647772692993_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6789060247306014343_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:34 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6789060247306014343_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6789060247306014343_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_5187050944088572182_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_5187050944088572182_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_5187050944088572182_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7075595753017259392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7075595753017259392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7075595753017259392_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_6879611049682927992_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_6879611049682927992_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_6879611049682927992_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7983957746726698450_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7983957746726698450_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:52:35 INFO HiveMetaStore: 11: get_database: trusted 23/06/28 07:52:35 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:52:35 INFO HiveMetaStore: 11: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:52:35 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:52:35 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:52:35 INFO HiveMetaStore: 11: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:52:35 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO HiveMetaStore: 12: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:52:36 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:52:36 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO ClusterLoadMonitor: Added query with execution ID:256. Current active queries:1 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:36 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:36 INFO HiveMetaStore: 2: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO HiveMetaStore: 2: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:52:36 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:52:36 INFO HiveMetaStore: 2: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO AbstractDatabricksHandler: dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:36 WARN InputFieldsCollector: Could not extract dataset identifier from org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier 23/06/28 07:52:36 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:36.827Z","run":{"runId":"0eb11396-af76-4bf3-a6fb-451e49ce1273","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.ReplaceTableAsSelect","num-children":2,"name":0,"partitioning":[],"query":1,"tableSpec":null,"writeOptions":null,"orCreate":true},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier","num-children":0,"catalog":null,"identifier":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_atomic_replace_table_as_select_warehouse_lms_lmsuu_mdl_groups_members_sql_merge","facets":{}},"inputs":[],"outputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"storage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/StorageDatasetFacet.json#/$defs/StorageDatasetFacet","storageLayer":"unity","fileFormat":"parquet"},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"default.LMS_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:36 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:36 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:37 INFO DeltaLog: Creating initial snapshot without metadata, because the directory is empty 23/06/28 07:52:37 INFO InitialSnapshot: [tableId=ae32fdd2-bfd7-4178-9476-147cb45f4958] Created snapshot InitialSnapshot(path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log, version=-1, metadata=Metadata(9f947d3e-6fbd-48fa-9011-9c44b5256eb8,null,null,Format(parquet,Map()),null,List(),Map(),Some(1687938757259)), logSegment=LogSegment(dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log,-1,List(),List(),None,-1), checksumOpt=None) 23/06/28 07:52:37 INFO DeltaLog: No delta log found for the Delta table at dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log 23/06/28 07:52:37 INFO InitialSnapshot: [tableId=9f947d3e-6fbd-48fa-9011-9c44b5256eb8] Created snapshot InitialSnapshot(path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log, version=-1, metadata=Metadata(e48355b8-0ee0-4734-8f1b-17582bd0a130,null,null,Format(parquet,Map()),null,List(),Map(),Some(1687938757301)), logSegment=LogSegment(dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log,-1,List(),List(),None,-1), checksumOpt=None) 23/06/28 07:52:37 INFO OptimisticTransaction: [tableId=e48355b8,txnId=46fb154c] Updated metadata from - to Metadata(0a243321-de86-4bc0-9520-53c494e03423,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},ArrayBuffer(),Map(),Some(1687938757305)) 23/06/28 07:52:37 INFO ParquetUtils: Using user defined output committer for Parquet: org.apache.spark.sql.parquet.DirectParquetOutputCommitter 23/06/28 07:52:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:37 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:37.457Z","run":{"runId":"408dc69b-1d8e-4cd3-bc0c-60a969efa6e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, sql]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:37 INFO CodeGenerator: Code generated in 7.110585 ms 23/06/28 07:52:37 INFO SparkContext: Starting job: write at WriteIntoDeltaCommand.scala:70 23/06/28 07:52:37 INFO DAGScheduler: Got job 81 (write at WriteIntoDeltaCommand.scala:70) with 192 output partitions 23/06/28 07:52:37 INFO DAGScheduler: Final stage: ResultStage 195 (write at WriteIntoDeltaCommand.scala:70) 23/06/28 07:52:37 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:52:37 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:37 INFO DAGScheduler: Submitting ResultStage 195 (MapPartitionsRDD[410] at write at WriteIntoDeltaCommand.scala:70), which has no missing parents 23/06/28 07:52:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:37 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:37 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:37 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:52:37 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:37 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:37 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:37 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:37 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:37 INFO DAGScheduler: Submitting 192 missing tasks from ResultStage 195 (MapPartitionsRDD[410] at write at WriteIntoDeltaCommand.scala:70) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)) 23/06/28 07:52:37 INFO TaskSchedulerImpl: Adding task set 195.0 with 192 tasks resource profile 0 23/06/28 07:52:37 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 6803957155303634069, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 6803957155303634069. Created 6803957155303634069 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:52:37 INFO FairSchedulableBuilder: Added task set TaskSet_195.0 tasks to pool 6803957155303634069 23/06/28 07:52:37 INFO TaskSetManager: Starting task 0.0 in stage 195.0 (TID 1375) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 1.0 in stage 195.0 (TID 1376) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 2.0 in stage 195.0 (TID 1377) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 3.0 in stage 195.0 (TID 1378) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 4.0 in stage 195.0 (TID 1379) (10.60.24.10, executor 0, partition 4, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 5.0 in stage 195.0 (TID 1380) (10.60.24.13, executor 1, partition 5, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 6.0 in stage 195.0 (TID 1381) (10.60.24.10, executor 0, partition 6, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 7.0 in stage 195.0 (TID 1382) (10.60.24.13, executor 1, partition 7, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 8.0 in stage 195.0 (TID 1383) (10.60.24.10, executor 0, partition 8, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 9.0 in stage 195.0 (TID 1384) (10.60.24.13, executor 1, partition 9, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 10.0 in stage 195.0 (TID 1385) (10.60.24.10, executor 0, partition 10, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 11.0 in stage 195.0 (TID 1386) (10.60.24.13, executor 1, partition 11, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 12.0 in stage 195.0 (TID 1387) (10.60.24.10, executor 0, partition 12, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 13.0 in stage 195.0 (TID 1388) (10.60.24.13, executor 1, partition 13, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 14.0 in stage 195.0 (TID 1389) (10.60.24.10, executor 0, partition 14, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 15.0 in stage 195.0 (TID 1390) (10.60.24.13, executor 1, partition 15, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 16.0 in stage 195.0 (TID 1391) (10.60.24.10, executor 0, partition 16, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 17.0 in stage 195.0 (TID 1392) (10.60.24.13, executor 1, partition 17, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 18.0 in stage 195.0 (TID 1393) (10.60.24.10, executor 0, partition 18, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 19.0 in stage 195.0 (TID 1394) (10.60.24.13, executor 1, partition 19, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 20.0 in stage 195.0 (TID 1395) (10.60.24.10, executor 0, partition 20, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 21.0 in stage 195.0 (TID 1396) (10.60.24.13, executor 1, partition 21, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 22.0 in stage 195.0 (TID 1397) (10.60.24.10, executor 0, partition 22, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 23.0 in stage 195.0 (TID 1398) (10.60.24.13, executor 1, partition 23, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 24.0 in stage 195.0 (TID 1399) (10.60.24.10, executor 0, partition 24, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 25.0 in stage 195.0 (TID 1400) (10.60.24.13, executor 1, partition 25, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 26.0 in stage 195.0 (TID 1401) (10.60.24.10, executor 0, partition 26, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 27.0 in stage 195.0 (TID 1402) (10.60.24.13, executor 1, partition 27, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 28.0 in stage 195.0 (TID 1403) (10.60.24.10, executor 0, partition 28, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 29.0 in stage 195.0 (TID 1404) (10.60.24.13, executor 1, partition 29, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 30.0 in stage 195.0 (TID 1405) (10.60.24.10, executor 0, partition 30, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 31.0 in stage 195.0 (TID 1406) (10.60.24.13, executor 1, partition 31, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 32.0 in stage 195.0 (TID 1407) (10.60.24.10, executor 0, partition 32, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 33.0 in stage 195.0 (TID 1408) (10.60.24.13, executor 1, partition 33, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 34.0 in stage 195.0 (TID 1409) (10.60.24.10, executor 0, partition 34, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 35.0 in stage 195.0 (TID 1410) (10.60.24.13, executor 1, partition 35, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 36.0 in stage 195.0 (TID 1411) (10.60.24.10, executor 0, partition 36, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 37.0 in stage 195.0 (TID 1412) (10.60.24.13, executor 1, partition 37, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 38.0 in stage 195.0 (TID 1413) (10.60.24.10, executor 0, partition 38, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 39.0 in stage 195.0 (TID 1414) (10.60.24.13, executor 1, partition 39, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 40.0 in stage 195.0 (TID 1415) (10.60.24.10, executor 0, partition 40, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 41.0 in stage 195.0 (TID 1416) (10.60.24.13, executor 1, partition 41, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 42.0 in stage 195.0 (TID 1417) (10.60.24.10, executor 0, partition 42, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 43.0 in stage 195.0 (TID 1418) (10.60.24.13, executor 1, partition 43, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 44.0 in stage 195.0 (TID 1419) (10.60.24.10, executor 0, partition 44, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 45.0 in stage 195.0 (TID 1420) (10.60.24.13, executor 1, partition 45, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 46.0 in stage 195.0 (TID 1421) (10.60.24.10, executor 0, partition 46, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 47.0 in stage 195.0 (TID 1422) (10.60.24.13, executor 1, partition 47, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 48.0 in stage 195.0 (TID 1423) (10.60.24.10, executor 0, partition 48, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 49.0 in stage 195.0 (TID 1424) (10.60.24.13, executor 1, partition 49, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 50.0 in stage 195.0 (TID 1425) (10.60.24.10, executor 0, partition 50, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 51.0 in stage 195.0 (TID 1426) (10.60.24.13, executor 1, partition 51, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 52.0 in stage 195.0 (TID 1427) (10.60.24.10, executor 0, partition 52, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 53.0 in stage 195.0 (TID 1428) (10.60.24.13, executor 1, partition 53, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 54.0 in stage 195.0 (TID 1429) (10.60.24.10, executor 0, partition 54, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 55.0 in stage 195.0 (TID 1430) (10.60.24.13, executor 1, partition 55, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 56.0 in stage 195.0 (TID 1431) (10.60.24.10, executor 0, partition 56, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 57.0 in stage 195.0 (TID 1432) (10.60.24.13, executor 1, partition 57, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 58.0 in stage 195.0 (TID 1433) (10.60.24.10, executor 0, partition 58, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 59.0 in stage 195.0 (TID 1434) (10.60.24.13, executor 1, partition 59, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 60.0 in stage 195.0 (TID 1435) (10.60.24.10, executor 0, partition 60, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 61.0 in stage 195.0 (TID 1436) (10.60.24.13, executor 1, partition 61, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 62.0 in stage 195.0 (TID 1437) (10.60.24.10, executor 0, partition 62, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 63.0 in stage 195.0 (TID 1438) (10.60.24.13, executor 1, partition 63, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 64.0 in stage 195.0 (TID 1439) (10.60.24.10, executor 0, partition 64, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 65.0 in stage 195.0 (TID 1440) (10.60.24.13, executor 1, partition 65, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 66.0 in stage 195.0 (TID 1441) (10.60.24.10, executor 0, partition 66, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 67.0 in stage 195.0 (TID 1442) (10.60.24.13, executor 1, partition 67, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 68.0 in stage 195.0 (TID 1443) (10.60.24.10, executor 0, partition 68, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 69.0 in stage 195.0 (TID 1444) (10.60.24.13, executor 1, partition 69, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 70.0 in stage 195.0 (TID 1445) (10.60.24.10, executor 0, partition 70, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 71.0 in stage 195.0 (TID 1446) (10.60.24.13, executor 1, partition 71, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 72.0 in stage 195.0 (TID 1447) (10.60.24.10, executor 0, partition 72, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 73.0 in stage 195.0 (TID 1448) (10.60.24.13, executor 1, partition 73, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 74.0 in stage 195.0 (TID 1449) (10.60.24.10, executor 0, partition 74, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 75.0 in stage 195.0 (TID 1450) (10.60.24.13, executor 1, partition 75, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 76.0 in stage 195.0 (TID 1451) (10.60.24.10, executor 0, partition 76, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 77.0 in stage 195.0 (TID 1452) (10.60.24.13, executor 1, partition 77, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 78.0 in stage 195.0 (TID 1453) (10.60.24.10, executor 0, partition 78, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 79.0 in stage 195.0 (TID 1454) (10.60.24.13, executor 1, partition 79, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 80.0 in stage 195.0 (TID 1455) (10.60.24.10, executor 0, partition 80, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 81.0 in stage 195.0 (TID 1456) (10.60.24.13, executor 1, partition 81, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 82.0 in stage 195.0 (TID 1457) (10.60.24.10, executor 0, partition 82, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 83.0 in stage 195.0 (TID 1458) (10.60.24.13, executor 1, partition 83, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 84.0 in stage 195.0 (TID 1459) (10.60.24.10, executor 0, partition 84, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 85.0 in stage 195.0 (TID 1460) (10.60.24.13, executor 1, partition 85, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 86.0 in stage 195.0 (TID 1461) (10.60.24.10, executor 0, partition 86, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 87.0 in stage 195.0 (TID 1462) (10.60.24.13, executor 1, partition 87, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 88.0 in stage 195.0 (TID 1463) (10.60.24.10, executor 0, partition 88, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 89.0 in stage 195.0 (TID 1464) (10.60.24.13, executor 1, partition 89, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 90.0 in stage 195.0 (TID 1465) (10.60.24.10, executor 0, partition 90, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 91.0 in stage 195.0 (TID 1466) (10.60.24.13, executor 1, partition 91, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 92.0 in stage 195.0 (TID 1467) (10.60.24.10, executor 0, partition 92, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 93.0 in stage 195.0 (TID 1468) (10.60.24.13, executor 1, partition 93, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 94.0 in stage 195.0 (TID 1469) (10.60.24.10, executor 0, partition 94, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 95.0 in stage 195.0 (TID 1470) (10.60.24.13, executor 1, partition 95, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 96.0 in stage 195.0 (TID 1471) (10.60.24.10, executor 0, partition 96, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 97.0 in stage 195.0 (TID 1472) (10.60.24.13, executor 1, partition 97, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 98.0 in stage 195.0 (TID 1473) (10.60.24.10, executor 0, partition 98, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 99.0 in stage 195.0 (TID 1474) (10.60.24.13, executor 1, partition 99, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 100.0 in stage 195.0 (TID 1475) (10.60.24.10, executor 0, partition 100, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 101.0 in stage 195.0 (TID 1476) (10.60.24.13, executor 1, partition 101, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 102.0 in stage 195.0 (TID 1477) (10.60.24.10, executor 0, partition 102, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 103.0 in stage 195.0 (TID 1478) (10.60.24.13, executor 1, partition 103, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 104.0 in stage 195.0 (TID 1479) (10.60.24.10, executor 0, partition 104, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 105.0 in stage 195.0 (TID 1480) (10.60.24.13, executor 1, partition 105, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 106.0 in stage 195.0 (TID 1481) (10.60.24.10, executor 0, partition 106, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 107.0 in stage 195.0 (TID 1482) (10.60.24.13, executor 1, partition 107, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 108.0 in stage 195.0 (TID 1483) (10.60.24.10, executor 0, partition 108, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 109.0 in stage 195.0 (TID 1484) (10.60.24.13, executor 1, partition 109, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 110.0 in stage 195.0 (TID 1485) (10.60.24.10, executor 0, partition 110, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 111.0 in stage 195.0 (TID 1486) (10.60.24.13, executor 1, partition 111, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 112.0 in stage 195.0 (TID 1487) (10.60.24.10, executor 0, partition 112, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 113.0 in stage 195.0 (TID 1488) (10.60.24.13, executor 1, partition 113, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 114.0 in stage 195.0 (TID 1489) (10.60.24.10, executor 0, partition 114, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 115.0 in stage 195.0 (TID 1490) (10.60.24.13, executor 1, partition 115, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 116.0 in stage 195.0 (TID 1491) (10.60.24.10, executor 0, partition 116, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 117.0 in stage 195.0 (TID 1492) (10.60.24.13, executor 1, partition 117, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 118.0 in stage 195.0 (TID 1493) (10.60.24.10, executor 0, partition 118, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 119.0 in stage 195.0 (TID 1494) (10.60.24.13, executor 1, partition 119, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 120.0 in stage 195.0 (TID 1495) (10.60.24.10, executor 0, partition 120, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 121.0 in stage 195.0 (TID 1496) (10.60.24.13, executor 1, partition 121, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 122.0 in stage 195.0 (TID 1497) (10.60.24.10, executor 0, partition 122, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 123.0 in stage 195.0 (TID 1498) (10.60.24.13, executor 1, partition 123, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 124.0 in stage 195.0 (TID 1499) (10.60.24.10, executor 0, partition 124, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 125.0 in stage 195.0 (TID 1500) (10.60.24.13, executor 1, partition 125, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 126.0 in stage 195.0 (TID 1501) (10.60.24.10, executor 0, partition 126, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 127.0 in stage 195.0 (TID 1502) (10.60.24.13, executor 1, partition 127, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 128.0 in stage 195.0 (TID 1503) (10.60.24.10, executor 0, partition 128, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 129.0 in stage 195.0 (TID 1504) (10.60.24.13, executor 1, partition 129, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 130.0 in stage 195.0 (TID 1505) (10.60.24.10, executor 0, partition 130, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 131.0 in stage 195.0 (TID 1506) (10.60.24.13, executor 1, partition 131, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 132.0 in stage 195.0 (TID 1507) (10.60.24.10, executor 0, partition 132, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 133.0 in stage 195.0 (TID 1508) (10.60.24.13, executor 1, partition 133, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 134.0 in stage 195.0 (TID 1509) (10.60.24.10, executor 0, partition 134, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 135.0 in stage 195.0 (TID 1510) (10.60.24.13, executor 1, partition 135, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 136.0 in stage 195.0 (TID 1511) (10.60.24.10, executor 0, partition 136, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 137.0 in stage 195.0 (TID 1512) (10.60.24.13, executor 1, partition 137, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 138.0 in stage 195.0 (TID 1513) (10.60.24.10, executor 0, partition 138, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 139.0 in stage 195.0 (TID 1514) (10.60.24.13, executor 1, partition 139, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 140.0 in stage 195.0 (TID 1515) (10.60.24.10, executor 0, partition 140, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 141.0 in stage 195.0 (TID 1516) (10.60.24.13, executor 1, partition 141, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 142.0 in stage 195.0 (TID 1517) (10.60.24.10, executor 0, partition 142, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 143.0 in stage 195.0 (TID 1518) (10.60.24.13, executor 1, partition 143, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 144.0 in stage 195.0 (TID 1519) (10.60.24.10, executor 0, partition 144, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 145.0 in stage 195.0 (TID 1520) (10.60.24.13, executor 1, partition 145, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 146.0 in stage 195.0 (TID 1521) (10.60.24.10, executor 0, partition 146, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 147.0 in stage 195.0 (TID 1522) (10.60.24.13, executor 1, partition 147, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 148.0 in stage 195.0 (TID 1523) (10.60.24.10, executor 0, partition 148, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 149.0 in stage 195.0 (TID 1524) (10.60.24.13, executor 1, partition 149, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 150.0 in stage 195.0 (TID 1525) (10.60.24.10, executor 0, partition 150, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 151.0 in stage 195.0 (TID 1526) (10.60.24.13, executor 1, partition 151, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 152.0 in stage 195.0 (TID 1527) (10.60.24.10, executor 0, partition 152, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 153.0 in stage 195.0 (TID 1528) (10.60.24.13, executor 1, partition 153, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 154.0 in stage 195.0 (TID 1529) (10.60.24.10, executor 0, partition 154, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 155.0 in stage 195.0 (TID 1530) (10.60.24.13, executor 1, partition 155, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 156.0 in stage 195.0 (TID 1531) (10.60.24.10, executor 0, partition 156, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 157.0 in stage 195.0 (TID 1532) (10.60.24.13, executor 1, partition 157, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 158.0 in stage 195.0 (TID 1533) (10.60.24.10, executor 0, partition 158, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 159.0 in stage 195.0 (TID 1534) (10.60.24.13, executor 1, partition 159, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 160.0 in stage 195.0 (TID 1535) (10.60.24.10, executor 0, partition 160, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 161.0 in stage 195.0 (TID 1536) (10.60.24.13, executor 1, partition 161, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 162.0 in stage 195.0 (TID 1537) (10.60.24.10, executor 0, partition 162, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 163.0 in stage 195.0 (TID 1538) (10.60.24.13, executor 1, partition 163, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 164.0 in stage 195.0 (TID 1539) (10.60.24.10, executor 0, partition 164, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 165.0 in stage 195.0 (TID 1540) (10.60.24.13, executor 1, partition 165, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 166.0 in stage 195.0 (TID 1541) (10.60.24.10, executor 0, partition 166, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 167.0 in stage 195.0 (TID 1542) (10.60.24.13, executor 1, partition 167, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 168.0 in stage 195.0 (TID 1543) (10.60.24.10, executor 0, partition 168, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 169.0 in stage 195.0 (TID 1544) (10.60.24.13, executor 1, partition 169, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 170.0 in stage 195.0 (TID 1545) (10.60.24.10, executor 0, partition 170, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 171.0 in stage 195.0 (TID 1546) (10.60.24.13, executor 1, partition 171, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 172.0 in stage 195.0 (TID 1547) (10.60.24.10, executor 0, partition 172, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 173.0 in stage 195.0 (TID 1548) (10.60.24.13, executor 1, partition 173, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 174.0 in stage 195.0 (TID 1549) (10.60.24.10, executor 0, partition 174, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 175.0 in stage 195.0 (TID 1550) (10.60.24.13, executor 1, partition 175, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 176.0 in stage 195.0 (TID 1551) (10.60.24.10, executor 0, partition 176, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 177.0 in stage 195.0 (TID 1552) (10.60.24.13, executor 1, partition 177, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 178.0 in stage 195.0 (TID 1553) (10.60.24.10, executor 0, partition 178, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 179.0 in stage 195.0 (TID 1554) (10.60.24.13, executor 1, partition 179, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 180.0 in stage 195.0 (TID 1555) (10.60.24.10, executor 0, partition 180, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 181.0 in stage 195.0 (TID 1556) (10.60.24.13, executor 1, partition 181, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 182.0 in stage 195.0 (TID 1557) (10.60.24.10, executor 0, partition 182, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 183.0 in stage 195.0 (TID 1558) (10.60.24.13, executor 1, partition 183, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 184.0 in stage 195.0 (TID 1559) (10.60.24.10, executor 0, partition 184, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 185.0 in stage 195.0 (TID 1560) (10.60.24.13, executor 1, partition 185, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 186.0 in stage 195.0 (TID 1561) (10.60.24.10, executor 0, partition 186, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 187.0 in stage 195.0 (TID 1562) (10.60.24.13, executor 1, partition 187, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 188.0 in stage 195.0 (TID 1563) (10.60.24.10, executor 0, partition 188, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 189.0 in stage 195.0 (TID 1564) (10.60.24.13, executor 1, partition 189, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 190.0 in stage 195.0 (TID 1565) (10.60.24.10, executor 0, partition 190, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO TaskSetManager: Starting task 191.0 in stage 195.0 (TID 1566) (10.60.24.13, executor 1, partition 191, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:37 INFO MemoryStore: Block broadcast_128 stored as values in memory (estimated size 272.2 KiB, free 750.6 GiB) 23/06/28 07:52:37 INFO MemoryStore: Block broadcast_128_piece0 stored as bytes in memory (estimated size 85.1 KiB, free 750.6 GiB) 23/06/28 07:52:37 INFO BlockManagerInfo: Added broadcast_128_piece0 in memory on 10.60.24.15:40507 (size: 85.1 KiB, free: 750.7 GiB) 23/06/28 07:52:37 INFO SparkContext: Created broadcast 128 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:37 INFO BlockManagerInfo: Added broadcast_128_piece0 in memory on 10.60.24.13:44035 (size: 85.1 KiB, free: 487.7 GiB) 23/06/28 07:52:37 INFO BlockManagerInfo: Added broadcast_128_piece0 in memory on 10.60.24.10:41041 (size: 85.1 KiB, free: 487.7 GiB) 23/06/28 07:52:37 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:37.479Z","run":{"runId":"408dc69b-1d8e-4cd3-bc0c-60a969efa6e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, sql]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532233","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/trusted/function/load-trusted-sqledw-v2","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"980829582242690","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:37 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:52:40 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:52:42 INFO TaskSetManager: Finished task 157.0 in stage 195.0 (TID 1532) in 5311 ms on 10.60.24.13 (executor 1) (1/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 161.0 in stage 195.0 (TID 1536) in 5311 ms on 10.60.24.13 (executor 1) (2/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 177.0 in stage 195.0 (TID 1552) in 5310 ms on 10.60.24.13 (executor 1) (3/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 91.0 in stage 195.0 (TID 1466) in 5312 ms on 10.60.24.13 (executor 1) (4/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 23.0 in stage 195.0 (TID 1398) in 5313 ms on 10.60.24.13 (executor 1) (5/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 71.0 in stage 195.0 (TID 1446) in 5313 ms on 10.60.24.13 (executor 1) (6/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 13.0 in stage 195.0 (TID 1388) in 5314 ms on 10.60.24.13 (executor 1) (7/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 127.0 in stage 195.0 (TID 1502) in 5312 ms on 10.60.24.13 (executor 1) (8/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 65.0 in stage 195.0 (TID 1440) in 5313 ms on 10.60.24.13 (executor 1) (9/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 175.0 in stage 195.0 (TID 1550) in 5311 ms on 10.60.24.13 (executor 1) (10/192) 23/06/28 07:52:42 INFO PythonAccumulatorV2: Connected to AccumulatorServer at host: 127.0.0.1 port: 58913 23/06/28 07:52:42 INFO TaskSetManager: Finished task 63.0 in stage 195.0 (TID 1438) in 5313 ms on 10.60.24.13 (executor 1) (11/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 183.0 in stage 195.0 (TID 1558) in 5311 ms on 10.60.24.13 (executor 1) (12/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 125.0 in stage 195.0 (TID 1500) in 5312 ms on 10.60.24.13 (executor 1) (13/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 49.0 in stage 195.0 (TID 1424) in 5314 ms on 10.60.24.13 (executor 1) (14/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 105.0 in stage 195.0 (TID 1480) in 5313 ms on 10.60.24.13 (executor 1) (15/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 149.0 in stage 195.0 (TID 1524) in 5313 ms on 10.60.24.13 (executor 1) (16/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 41.0 in stage 195.0 (TID 1416) in 5315 ms on 10.60.24.13 (executor 1) (17/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 47.0 in stage 195.0 (TID 1422) in 5315 ms on 10.60.24.13 (executor 1) (18/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 151.0 in stage 195.0 (TID 1526) in 5313 ms on 10.60.24.13 (executor 1) (19/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 181.0 in stage 195.0 (TID 1556) in 5312 ms on 10.60.24.13 (executor 1) (20/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 1.0 in stage 195.0 (TID 1376) in 5316 ms on 10.60.24.13 (executor 1) (21/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 109.0 in stage 195.0 (TID 1484) in 5313 ms on 10.60.24.13 (executor 1) (22/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 83.0 in stage 195.0 (TID 1458) in 5315 ms on 10.60.24.13 (executor 1) (23/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 19.0 in stage 195.0 (TID 1394) in 5319 ms on 10.60.24.13 (executor 1) (24/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 73.0 in stage 195.0 (TID 1448) in 5319 ms on 10.60.24.13 (executor 1) (25/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 51.0 in stage 195.0 (TID 1426) in 5320 ms on 10.60.24.13 (executor 1) (26/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 141.0 in stage 195.0 (TID 1516) in 5319 ms on 10.60.24.13 (executor 1) (27/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 119.0 in stage 195.0 (TID 1494) in 5321 ms on 10.60.24.13 (executor 1) (28/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 11.0 in stage 195.0 (TID 1386) in 5324 ms on 10.60.24.13 (executor 1) (29/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 89.0 in stage 195.0 (TID 1464) in 5325 ms on 10.60.24.13 (executor 1) (30/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 153.0 in stage 195.0 (TID 1528) in 5324 ms on 10.60.24.13 (executor 1) (31/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 137.0 in stage 195.0 (TID 1512) in 5324 ms on 10.60.24.13 (executor 1) (32/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 95.0 in stage 195.0 (TID 1470) in 5325 ms on 10.60.24.13 (executor 1) (33/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 31.0 in stage 195.0 (TID 1406) in 5328 ms on 10.60.24.13 (executor 1) (34/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 93.0 in stage 195.0 (TID 1468) in 5327 ms on 10.60.24.13 (executor 1) (35/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 189.0 in stage 195.0 (TID 1564) in 5326 ms on 10.60.24.13 (executor 1) (36/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 69.0 in stage 195.0 (TID 1444) in 5328 ms on 10.60.24.13 (executor 1) (37/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 35.0 in stage 195.0 (TID 1410) in 5330 ms on 10.60.24.13 (executor 1) (38/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 133.0 in stage 195.0 (TID 1508) in 5329 ms on 10.60.24.13 (executor 1) (39/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 145.0 in stage 195.0 (TID 1520) in 5331 ms on 10.60.24.13 (executor 1) (40/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 81.0 in stage 195.0 (TID 1456) in 5333 ms on 10.60.24.13 (executor 1) (41/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 39.0 in stage 195.0 (TID 1414) in 5334 ms on 10.60.24.13 (executor 1) (42/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 61.0 in stage 195.0 (TID 1436) in 5335 ms on 10.60.24.13 (executor 1) (43/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 75.0 in stage 195.0 (TID 1450) in 5336 ms on 10.60.24.13 (executor 1) (44/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 185.0 in stage 195.0 (TID 1560) in 5335 ms on 10.60.24.13 (executor 1) (45/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 55.0 in stage 195.0 (TID 1430) in 5338 ms on 10.60.24.13 (executor 1) (46/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 57.0 in stage 195.0 (TID 1432) in 5340 ms on 10.60.24.13 (executor 1) (47/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 97.0 in stage 195.0 (TID 1472) in 5339 ms on 10.60.24.13 (executor 1) (48/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 129.0 in stage 195.0 (TID 1504) in 5339 ms on 10.60.24.13 (executor 1) (49/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 147.0 in stage 195.0 (TID 1522) in 5339 ms on 10.60.24.13 (executor 1) (50/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 167.0 in stage 195.0 (TID 1542) in 5339 ms on 10.60.24.13 (executor 1) (51/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 171.0 in stage 195.0 (TID 1546) in 5340 ms on 10.60.24.13 (executor 1) (52/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 159.0 in stage 195.0 (TID 1534) in 5342 ms on 10.60.24.13 (executor 1) (53/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 15.0 in stage 195.0 (TID 1390) in 5344 ms on 10.60.24.13 (executor 1) (54/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 7.0 in stage 195.0 (TID 1382) in 5345 ms on 10.60.24.13 (executor 1) (55/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 5.0 in stage 195.0 (TID 1380) in 5345 ms on 10.60.24.13 (executor 1) (56/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 173.0 in stage 195.0 (TID 1548) in 5343 ms on 10.60.24.13 (executor 1) (57/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 163.0 in stage 195.0 (TID 1538) in 5345 ms on 10.60.24.13 (executor 1) (58/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 99.0 in stage 195.0 (TID 1474) in 5349 ms on 10.60.24.13 (executor 1) (59/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 165.0 in stage 195.0 (TID 1540) in 5347 ms on 10.60.24.13 (executor 1) (60/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 3.0 in stage 195.0 (TID 1378) in 5351 ms on 10.60.24.13 (executor 1) (61/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 103.0 in stage 195.0 (TID 1478) in 5349 ms on 10.60.24.13 (executor 1) (62/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 131.0 in stage 195.0 (TID 1506) in 5348 ms on 10.60.24.13 (executor 1) (63/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 187.0 in stage 195.0 (TID 1562) in 5347 ms on 10.60.24.13 (executor 1) (64/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 53.0 in stage 195.0 (TID 1428) in 5350 ms on 10.60.24.13 (executor 1) (65/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 9.0 in stage 195.0 (TID 1384) in 5352 ms on 10.60.24.13 (executor 1) (66/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 85.0 in stage 195.0 (TID 1460) in 5350 ms on 10.60.24.13 (executor 1) (67/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 25.0 in stage 195.0 (TID 1400) in 5351 ms on 10.60.24.13 (executor 1) (68/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 33.0 in stage 195.0 (TID 1408) in 5351 ms on 10.60.24.13 (executor 1) (69/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 29.0 in stage 195.0 (TID 1404) in 5351 ms on 10.60.24.13 (executor 1) (70/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 37.0 in stage 195.0 (TID 1412) in 5351 ms on 10.60.24.13 (executor 1) (71/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 139.0 in stage 195.0 (TID 1514) in 5349 ms on 10.60.24.13 (executor 1) (72/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 143.0 in stage 195.0 (TID 1518) in 5349 ms on 10.60.24.13 (executor 1) (73/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 21.0 in stage 195.0 (TID 1396) in 5351 ms on 10.60.24.13 (executor 1) (74/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 27.0 in stage 195.0 (TID 1402) in 5352 ms on 10.60.24.13 (executor 1) (75/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 43.0 in stage 195.0 (TID 1418) in 5352 ms on 10.60.24.13 (executor 1) (76/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 123.0 in stage 195.0 (TID 1498) in 5350 ms on 10.60.24.13 (executor 1) (77/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 67.0 in stage 195.0 (TID 1442) in 5351 ms on 10.60.24.13 (executor 1) (78/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 17.0 in stage 195.0 (TID 1392) in 5352 ms on 10.60.24.13 (executor 1) (79/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 121.0 in stage 195.0 (TID 1496) in 5350 ms on 10.60.24.13 (executor 1) (80/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 77.0 in stage 195.0 (TID 1452) in 5351 ms on 10.60.24.13 (executor 1) (81/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 115.0 in stage 195.0 (TID 1490) in 5350 ms on 10.60.24.13 (executor 1) (82/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 135.0 in stage 195.0 (TID 1510) in 5350 ms on 10.60.24.13 (executor 1) (83/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 79.0 in stage 195.0 (TID 1454) in 5351 ms on 10.60.24.13 (executor 1) (84/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 87.0 in stage 195.0 (TID 1462) in 5352 ms on 10.60.24.13 (executor 1) (85/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 107.0 in stage 195.0 (TID 1482) in 5352 ms on 10.60.24.13 (executor 1) (86/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 155.0 in stage 195.0 (TID 1530) in 5351 ms on 10.60.24.13 (executor 1) (87/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 113.0 in stage 195.0 (TID 1488) in 5351 ms on 10.60.24.13 (executor 1) (88/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 59.0 in stage 195.0 (TID 1434) in 5352 ms on 10.60.24.13 (executor 1) (89/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 179.0 in stage 195.0 (TID 1554) in 5350 ms on 10.60.24.13 (executor 1) (90/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 45.0 in stage 195.0 (TID 1420) in 5353 ms on 10.60.24.13 (executor 1) (91/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 169.0 in stage 195.0 (TID 1544) in 5350 ms on 10.60.24.13 (executor 1) (92/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 111.0 in stage 195.0 (TID 1486) in 5351 ms on 10.60.24.13 (executor 1) (93/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 117.0 in stage 195.0 (TID 1492) in 5351 ms on 10.60.24.13 (executor 1) (94/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 101.0 in stage 195.0 (TID 1476) in 5352 ms on 10.60.24.13 (executor 1) (95/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 44.0 in stage 195.0 (TID 1419) in 5428 ms on 10.60.24.10 (executor 0) (96/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 120.0 in stage 195.0 (TID 1495) in 5426 ms on 10.60.24.10 (executor 0) (97/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 66.0 in stage 195.0 (TID 1441) in 5427 ms on 10.60.24.10 (executor 0) (98/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 88.0 in stage 195.0 (TID 1463) in 5428 ms on 10.60.24.10 (executor 0) (99/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 148.0 in stage 195.0 (TID 1523) in 5442 ms on 10.60.24.10 (executor 0) (100/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 82.0 in stage 195.0 (TID 1457) in 5444 ms on 10.60.24.10 (executor 0) (101/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 156.0 in stage 195.0 (TID 1531) in 5443 ms on 10.60.24.10 (executor 0) (102/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 154.0 in stage 195.0 (TID 1529) in 5443 ms on 10.60.24.10 (executor 0) (103/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 6.0 in stage 195.0 (TID 1381) in 5446 ms on 10.60.24.10 (executor 0) (104/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 152.0 in stage 195.0 (TID 1527) in 5443 ms on 10.60.24.10 (executor 0) (105/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 4.0 in stage 195.0 (TID 1379) in 5447 ms on 10.60.24.10 (executor 0) (106/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 166.0 in stage 195.0 (TID 1541) in 5443 ms on 10.60.24.10 (executor 0) (107/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 118.0 in stage 195.0 (TID 1493) in 5447 ms on 10.60.24.10 (executor 0) (108/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 126.0 in stage 195.0 (TID 1501) in 5447 ms on 10.60.24.10 (executor 0) (109/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 182.0 in stage 195.0 (TID 1557) in 5446 ms on 10.60.24.10 (executor 0) (110/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 80.0 in stage 195.0 (TID 1455) in 5449 ms on 10.60.24.10 (executor 0) (111/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 70.0 in stage 195.0 (TID 1445) in 5449 ms on 10.60.24.10 (executor 0) (112/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 124.0 in stage 195.0 (TID 1499) in 5448 ms on 10.60.24.10 (executor 0) (113/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 48.0 in stage 195.0 (TID 1423) in 5450 ms on 10.60.24.10 (executor 0) (114/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 94.0 in stage 195.0 (TID 1469) in 5449 ms on 10.60.24.10 (executor 0) (115/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 58.0 in stage 195.0 (TID 1433) in 5450 ms on 10.60.24.10 (executor 0) (116/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 76.0 in stage 195.0 (TID 1451) in 5450 ms on 10.60.24.10 (executor 0) (117/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 68.0 in stage 195.0 (TID 1443) in 5450 ms on 10.60.24.10 (executor 0) (118/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 72.0 in stage 195.0 (TID 1447) in 5450 ms on 10.60.24.10 (executor 0) (119/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 22.0 in stage 195.0 (TID 1397) in 5451 ms on 10.60.24.10 (executor 0) (120/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 128.0 in stage 195.0 (TID 1503) in 5449 ms on 10.60.24.10 (executor 0) (121/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 100.0 in stage 195.0 (TID 1475) in 5450 ms on 10.60.24.10 (executor 0) (122/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 140.0 in stage 195.0 (TID 1515) in 5450 ms on 10.60.24.10 (executor 0) (123/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 184.0 in stage 195.0 (TID 1559) in 5449 ms on 10.60.24.10 (executor 0) (124/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 32.0 in stage 195.0 (TID 1407) in 5452 ms on 10.60.24.10 (executor 0) (125/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 172.0 in stage 195.0 (TID 1547) in 5449 ms on 10.60.24.10 (executor 0) (126/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 92.0 in stage 195.0 (TID 1467) in 5455 ms on 10.60.24.10 (executor 0) (127/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 188.0 in stage 195.0 (TID 1563) in 5453 ms on 10.60.24.10 (executor 0) (128/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 116.0 in stage 195.0 (TID 1491) in 5454 ms on 10.60.24.10 (executor 0) (129/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 162.0 in stage 195.0 (TID 1537) in 5455 ms on 10.60.24.10 (executor 0) (130/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 130.0 in stage 195.0 (TID 1505) in 5455 ms on 10.60.24.10 (executor 0) (131/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 26.0 in stage 195.0 (TID 1401) in 5457 ms on 10.60.24.10 (executor 0) (132/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 28.0 in stage 195.0 (TID 1403) in 5457 ms on 10.60.24.10 (executor 0) (133/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 60.0 in stage 195.0 (TID 1435) in 5456 ms on 10.60.24.10 (executor 0) (134/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 34.0 in stage 195.0 (TID 1409) in 5457 ms on 10.60.24.10 (executor 0) (135/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 54.0 in stage 195.0 (TID 1429) in 5458 ms on 10.60.24.10 (executor 0) (136/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 2.0 in stage 195.0 (TID 1377) in 5459 ms on 10.60.24.10 (executor 0) (137/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 50.0 in stage 195.0 (TID 1425) in 5458 ms on 10.60.24.10 (executor 0) (138/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 122.0 in stage 195.0 (TID 1497) in 5456 ms on 10.60.24.10 (executor 0) (139/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 36.0 in stage 195.0 (TID 1411) in 5458 ms on 10.60.24.10 (executor 0) (140/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 30.0 in stage 195.0 (TID 1405) in 5458 ms on 10.60.24.10 (executor 0) (141/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 46.0 in stage 195.0 (TID 1421) in 5458 ms on 10.60.24.10 (executor 0) (142/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 112.0 in stage 195.0 (TID 1487) in 5456 ms on 10.60.24.10 (executor 0) (143/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 56.0 in stage 195.0 (TID 1431) in 5459 ms on 10.60.24.10 (executor 0) (144/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 62.0 in stage 195.0 (TID 1437) in 5458 ms on 10.60.24.10 (executor 0) (145/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 176.0 in stage 195.0 (TID 1551) in 5456 ms on 10.60.24.10 (executor 0) (146/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 90.0 in stage 195.0 (TID 1465) in 5458 ms on 10.60.24.10 (executor 0) (147/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 84.0 in stage 195.0 (TID 1459) in 5458 ms on 10.60.24.10 (executor 0) (148/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 64.0 in stage 195.0 (TID 1439) in 5458 ms on 10.60.24.10 (executor 0) (149/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 178.0 in stage 195.0 (TID 1553) in 5456 ms on 10.60.24.10 (executor 0) (150/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 10.0 in stage 195.0 (TID 1385) in 5460 ms on 10.60.24.10 (executor 0) (151/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 190.0 in stage 195.0 (TID 1565) in 5457 ms on 10.60.24.10 (executor 0) (152/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 170.0 in stage 195.0 (TID 1545) in 5457 ms on 10.60.24.10 (executor 0) (153/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 144.0 in stage 195.0 (TID 1519) in 5458 ms on 10.60.24.10 (executor 0) (154/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 158.0 in stage 195.0 (TID 1533) in 5458 ms on 10.60.24.10 (executor 0) (155/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 146.0 in stage 195.0 (TID 1521) in 5458 ms on 10.60.24.10 (executor 0) (156/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 40.0 in stage 195.0 (TID 1415) in 5460 ms on 10.60.24.10 (executor 0) (157/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 98.0 in stage 195.0 (TID 1473) in 5459 ms on 10.60.24.10 (executor 0) (158/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 52.0 in stage 195.0 (TID 1427) in 5461 ms on 10.60.24.10 (executor 0) (159/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 174.0 in stage 195.0 (TID 1549) in 5458 ms on 10.60.24.10 (executor 0) (160/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 186.0 in stage 195.0 (TID 1561) in 5458 ms on 10.60.24.10 (executor 0) (161/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 14.0 in stage 195.0 (TID 1389) in 5461 ms on 10.60.24.10 (executor 0) (162/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 12.0 in stage 195.0 (TID 1387) in 5461 ms on 10.60.24.10 (executor 0) (163/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 38.0 in stage 195.0 (TID 1413) in 5461 ms on 10.60.24.10 (executor 0) (164/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 136.0 in stage 195.0 (TID 1511) in 5459 ms on 10.60.24.10 (executor 0) (165/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 24.0 in stage 195.0 (TID 1399) in 5461 ms on 10.60.24.10 (executor 0) (166/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 106.0 in stage 195.0 (TID 1481) in 5461 ms on 10.60.24.10 (executor 0) (167/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 108.0 in stage 195.0 (TID 1483) in 5460 ms on 10.60.24.10 (executor 0) (168/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 74.0 in stage 195.0 (TID 1449) in 5461 ms on 10.60.24.10 (executor 0) (169/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 96.0 in stage 195.0 (TID 1471) in 5461 ms on 10.60.24.10 (executor 0) (170/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 150.0 in stage 195.0 (TID 1525) in 5460 ms on 10.60.24.10 (executor 0) (171/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 138.0 in stage 195.0 (TID 1513) in 5460 ms on 10.60.24.10 (executor 0) (172/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 86.0 in stage 195.0 (TID 1461) in 5461 ms on 10.60.24.10 (executor 0) (173/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 114.0 in stage 195.0 (TID 1489) in 5461 ms on 10.60.24.10 (executor 0) (174/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 8.0 in stage 195.0 (TID 1383) in 5464 ms on 10.60.24.10 (executor 0) (175/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 110.0 in stage 195.0 (TID 1485) in 5461 ms on 10.60.24.10 (executor 0) (176/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 132.0 in stage 195.0 (TID 1507) in 5461 ms on 10.60.24.10 (executor 0) (177/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 20.0 in stage 195.0 (TID 1395) in 5463 ms on 10.60.24.10 (executor 0) (178/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 104.0 in stage 195.0 (TID 1479) in 5462 ms on 10.60.24.10 (executor 0) (179/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 164.0 in stage 195.0 (TID 1539) in 5461 ms on 10.60.24.10 (executor 0) (180/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 168.0 in stage 195.0 (TID 1543) in 5461 ms on 10.60.24.10 (executor 0) (181/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 134.0 in stage 195.0 (TID 1509) in 5462 ms on 10.60.24.10 (executor 0) (182/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 18.0 in stage 195.0 (TID 1393) in 5464 ms on 10.60.24.10 (executor 0) (183/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 180.0 in stage 195.0 (TID 1555) in 5461 ms on 10.60.24.10 (executor 0) (184/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 142.0 in stage 195.0 (TID 1517) in 5462 ms on 10.60.24.10 (executor 0) (185/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 78.0 in stage 195.0 (TID 1453) in 5463 ms on 10.60.24.10 (executor 0) (186/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 42.0 in stage 195.0 (TID 1417) in 5464 ms on 10.60.24.10 (executor 0) (187/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 0.0 in stage 195.0 (TID 1375) in 5466 ms on 10.60.24.10 (executor 0) (188/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 102.0 in stage 195.0 (TID 1477) in 5464 ms on 10.60.24.10 (executor 0) (189/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 160.0 in stage 195.0 (TID 1535) in 5463 ms on 10.60.24.10 (executor 0) (190/192) 23/06/28 07:52:42 INFO TaskSetManager: Finished task 16.0 in stage 195.0 (TID 1391) in 5465 ms on 10.60.24.10 (executor 0) (191/192) 23/06/28 07:52:43 INFO TaskSetManager: Finished task 191.0 in stage 195.0 (TID 1566) in 5580 ms on 10.60.24.13 (executor 1) (192/192) 23/06/28 07:52:43 INFO TaskSchedulerImpl: Removed TaskSet 195.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:43 INFO DAGScheduler: ResultStage 195 (write at WriteIntoDeltaCommand.scala:70) finished in 5.598 s 23/06/28 07:52:43 INFO DAGScheduler: Job 81 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:52:43 INFO TaskSchedulerImpl: Killing all running tasks in stage 195: Stage finished 23/06/28 07:52:43 INFO DAGScheduler: Job 81 finished: write at WriteIntoDeltaCommand.scala:70, took 5.601167 s 23/06/28 07:52:43 INFO FileFormatWriter: Start to commit write Job 13629f18-817e-400d-ab61-867955136ba1. 23/06/28 07:52:43 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:52:43 INFO FileFormatWriter: Write Job 13629f18-817e-400d-ab61-867955136ba1 committed. Elapsed time: 2 ms. 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO FileFormatWriter: Finished processing stats for write job 13629f18-817e-400d-ab61-867955136ba1. 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:43.078Z","run":{"runId":"408dc69b-1d8e-4cd3-bc0c-60a969efa6e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, sql]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO SparkContext: Starting job: collect at TransactionalWriteEdge.scala:576 23/06/28 07:52:43 INFO DAGScheduler: Got job 82 (collect at TransactionalWriteEdge.scala:576) with 1 output partitions 23/06/28 07:52:43 INFO DAGScheduler: Final stage: ResultStage 196 (collect at TransactionalWriteEdge.scala:576) 23/06/28 07:52:43 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:52:43 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:43 INFO DAGScheduler: Submitting ResultStage 196 (SQLExecutionRDD[412] at toRdd at TransactionalWriteEdge.scala:576), which has no missing parents 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:43 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:43 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:43 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 196 (SQLExecutionRDD[412] at toRdd at TransactionalWriteEdge.scala:576) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:52:43 INFO TaskSchedulerImpl: Adding task set 196.0 with 1 tasks resource profile 0 23/06/28 07:52:43 INFO FairSchedulableBuilder: Added task set TaskSet_196.0 tasks to pool 6803957155303634069 23/06/28 07:52:43 INFO TaskSetManager: Starting task 0.0 in stage 196.0 (TID 1567) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO MemoryStore: Block broadcast_129 stored as values in memory (estimated size 148.7 KiB, free 750.6 GiB) 23/06/28 07:52:43 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:43 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:52:43 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:43 INFO MemoryStore: Block broadcast_129_piece0 stored as bytes in memory (estimated size 29.9 KiB, free 750.6 GiB) 23/06/28 07:52:43 INFO BlockManagerInfo: Added broadcast_129_piece0 in memory on 10.60.24.15:40507 (size: 29.9 KiB, free: 750.7 GiB) 23/06/28 07:52:43 INFO SparkContext: Created broadcast 129 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:43 INFO BlockManagerInfo: Added broadcast_129_piece0 in memory on 10.60.24.10:41041 (size: 29.9 KiB, free: 487.7 GiB) 23/06/28 07:52:43 INFO TaskSetManager: Finished task 0.0 in stage 196.0 (TID 1567) in 14 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:52:43 INFO TaskSchedulerImpl: Removed TaskSet 196.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:43 INFO DAGScheduler: ResultStage 196 (collect at TransactionalWriteEdge.scala:576) finished in 0.016 s 23/06/28 07:52:43 INFO DAGScheduler: Job 82 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:52:43 INFO TaskSchedulerImpl: Killing all running tasks in stage 196: Stage finished 23/06/28 07:52:43 INFO DAGScheduler: Job 82 finished: collect at TransactionalWriteEdge.scala:576, took 0.017984 s 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:43.086Z","run":{"runId":"408dc69b-1d8e-4cd3-bc0c-60a969efa6e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, sql]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532233","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/trusted/function/load-trusted-sqledw-v2","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"980829582242690","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO SQLExecution: Received onJobStart for query with unregistered executionId 257 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:43.104Z","run":{"runId":"408dc69b-1d8e-4cd3-bc0c-60a969efa6e4","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"com.databricks.sql.transaction.tahoe.commands.WriteIntoDeltaCommand","num-children":1,"query":0,"outputColumnNames":"[id, sql]","outputSpec":null,"deltaLog":null,"metadata":null,"partitionColExprIds":[],"committer":null,"options":null,"hadoopConf":null,"statsTrackers":null,"overridingConfs":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_write_into_delta_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:43.17Z","run":{"runId":"c5468f79-5bf8-4a4c-9636-8102ff8a4c31","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"path","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"partitionValues","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"size","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"modificationTime","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"dataChange","dataType":"boolean","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"stats","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6035,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"tags","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"deletionVector","dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SerializeFromObject","num-children":1,"serializer":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"path","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"path","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"partitionValues","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"partitionValues","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"size","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"size","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"modificationTime","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"modificationTime","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"dataChange","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"dataChange","dataType":"boolean","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"tags","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"tags","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"deletionVector","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]}},{"class":"org.apache.spark.sql.catalyst.expressions.CreateNamedStruct","num-children":10,"children":[0,1,2,3,4,5,6,7,8,9]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"storageType","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"storageType","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"pathOrInlineDv","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"pathOrInlineDv","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"offset","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.UnwrapOption","num-children":1,"dataType":"integer","child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"offset","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"sizeInBytes","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"sizeInBytes","dataType":"integer","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"cardinality","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"cardinality","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.MapElements","num-children":1,"func":null,"argumentClass":"scala.Tuple1","argumentSchema":{"type":"struct","fields":[{"name":"_1","type":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}}]},"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6026,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeserializeToObject","num-children":1,"deserializer":[{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":1,"cls":"scala.Tuple1","arguments":[0],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":8,"cls":"com.databricks.sql.transaction.tahoe.actions.AddFile","arguments":[0,1,2,3,4,5,6,7],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"path"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"partitionValues"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"size\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"size"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"modificationTime\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"modificationTime"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Boolean\", name: \"dataChange\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"dataChange"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":5,"name":"stats"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":6,"name":"tags"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":5,"cls":"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor","arguments":[0,1,2,3,4],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"storageType"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"pathOrInlineDv"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.WrapOption","num-children":1,"child":0,"optType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"offset"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Int\", name: \"sizeInBytes\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"sizeInBytes"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"cardinality\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"cardinality"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6025,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.IsNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"txn","dataType":{"type":"struct","fields":[{"name":"appId","type":"string","nullable":true,"metadata":{}},{"name":"version","type":"long","nullable":false,"metadata":{}},{"name":"lastUpdated","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6009,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"remove","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"deletionTimestamp","type":"long","nullable":true,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"extendedFileMetadata","type":"boolean","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6011,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"metaData","dataType":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"description","type":"string","nullable":true,"metadata":{}},{"name":"format","type":{"type":"struct","fields":[{"name":"provider","type":"string","nullable":true,"metadata":{}},{"name":"options","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"schemaString","type":"string","nullable":true,"metadata":{}},{"name":"partitionColumns","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"configuration","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"createdTime","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6012,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"protocol","dataType":{"type":"struct","fields":[{"name":"minReaderVersion","type":"integer","nullable":false,"metadata":{}},{"name":"minWriterVersion","type":"integer","nullable":false,"metadata":{}},{"name":"readerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"writerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6013,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"cdc","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6014,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"commitInfo","dataType":{"type":"struct","fields":[{"name":"version","type":"long","nullable":true,"metadata":{}},{"name":"timestamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"userId","type":"string","nullable":true,"metadata":{}},{"name":"userName","type":"string","nullable":true,"metadata":{}},{"name":"operation","type":"string","nullable":true,"metadata":{}},{"name":"operationParameters","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"job","type":{"type":"struct","fields":[{"name":"jobId","type":"string","nullable":true,"metadata":{}},{"name":"jobName","type":"string","nullable":true,"metadata":{}},{"name":"runId","type":"string","nullable":true,"metadata":{}},{"name":"jobOwnerId","type":"string","nullable":true,"metadata":{}},{"name":"triggerType","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"notebook","type":{"type":"struct","fields":[{"name":"notebookId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"clusterId","type":"string","nullable":true,"metadata":{}},{"name":"readVersion","type":"long","nullable":true,"metadata":{}},{"name":"isolationLevel","type":"string","nullable":true,"metadata":{}},{"name":"isBlindAppend","type":"boolean","nullable":true,"metadata":{}},{"name":"operationMetrics","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"userMetadata","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"engineInfo","type":"string","nullable":true,"metadata":{}},{"name":"txnId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6015,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_project","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO CodeGenerator: Code generated in 64.610193 ms 23/06/28 07:52:43 INFO SparkContext: Starting job: executeCollect at DataSkippingReaderEdge.scala:118 23/06/28 07:52:43 INFO DAGScheduler: Job 83 finished: executeCollect at DataSkippingReaderEdge.scala:118, took 0.000210 s 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:52:43 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:52:43 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:52:43 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:52:43 INFO OptimisticTransaction: [tableId=e48355b8,txnId=46fb154c] Attempting to commit version 0 with 4 actions with WriteSerializable isolation level 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:43.272Z","run":{"runId":"c5468f79-5bf8-4a4c-9636-8102ff8a4c31","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"path","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"partitionValues","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"size","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"modificationTime","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"dataChange","dataType":"boolean","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"stats","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6035,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"tags","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"deletionVector","dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SerializeFromObject","num-children":1,"serializer":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"path","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"path","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"partitionValues","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"partitionValues","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"size","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"size","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"modificationTime","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"modificationTime","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"dataChange","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"dataChange","dataType":"boolean","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"tags","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"tags","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"deletionVector","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]}},{"class":"org.apache.spark.sql.catalyst.expressions.CreateNamedStruct","num-children":10,"children":[0,1,2,3,4,5,6,7,8,9]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"storageType","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"storageType","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"pathOrInlineDv","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"pathOrInlineDv","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"offset","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.UnwrapOption","num-children":1,"dataType":"integer","child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"offset","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"sizeInBytes","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"sizeInBytes","dataType":"integer","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"cardinality","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"cardinality","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.MapElements","num-children":1,"func":null,"argumentClass":"scala.Tuple1","argumentSchema":{"type":"struct","fields":[{"name":"_1","type":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}}]},"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6026,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeserializeToObject","num-children":1,"deserializer":[{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":1,"cls":"scala.Tuple1","arguments":[0],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":8,"cls":"com.databricks.sql.transaction.tahoe.actions.AddFile","arguments":[0,1,2,3,4,5,6,7],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"path"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"partitionValues"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"size\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"size"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"modificationTime\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"modificationTime"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Boolean\", name: \"dataChange\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"dataChange"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":5,"name":"stats"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":6,"name":"tags"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":5,"cls":"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor","arguments":[0,1,2,3,4],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"storageType"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"pathOrInlineDv"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.WrapOption","num-children":1,"child":0,"optType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"offset"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Int\", name: \"sizeInBytes\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"sizeInBytes"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"cardinality\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"cardinality"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6025,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.IsNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"txn","dataType":{"type":"struct","fields":[{"name":"appId","type":"string","nullable":true,"metadata":{}},{"name":"version","type":"long","nullable":false,"metadata":{}},{"name":"lastUpdated","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6009,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"remove","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"deletionTimestamp","type":"long","nullable":true,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"extendedFileMetadata","type":"boolean","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6011,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"metaData","dataType":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"description","type":"string","nullable":true,"metadata":{}},{"name":"format","type":{"type":"struct","fields":[{"name":"provider","type":"string","nullable":true,"metadata":{}},{"name":"options","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"schemaString","type":"string","nullable":true,"metadata":{}},{"name":"partitionColumns","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"configuration","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"createdTime","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6012,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"protocol","dataType":{"type":"struct","fields":[{"name":"minReaderVersion","type":"integer","nullable":false,"metadata":{}},{"name":"minWriterVersion","type":"integer","nullable":false,"metadata":{}},{"name":"readerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"writerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6013,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"cdc","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6014,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"commitInfo","dataType":{"type":"struct","fields":[{"name":"version","type":"long","nullable":true,"metadata":{}},{"name":"timestamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"userId","type":"string","nullable":true,"metadata":{}},{"name":"userName","type":"string","nullable":true,"metadata":{}},{"name":"operation","type":"string","nullable":true,"metadata":{}},{"name":"operationParameters","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"job","type":{"type":"struct","fields":[{"name":"jobId","type":"string","nullable":true,"metadata":{}},{"name":"jobName","type":"string","nullable":true,"metadata":{}},{"name":"runId","type":"string","nullable":true,"metadata":{}},{"name":"jobOwnerId","type":"string","nullable":true,"metadata":{}},{"name":"triggerType","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"notebook","type":{"type":"struct","fields":[{"name":"notebookId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"clusterId","type":"string","nullable":true,"metadata":{}},{"name":"readVersion","type":"long","nullable":true,"metadata":{}},{"name":"isolationLevel","type":"string","nullable":true,"metadata":{}},{"name":"isBlindAppend","type":"boolean","nullable":true,"metadata":{}},{"name":"operationMetrics","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"userMetadata","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"engineInfo","type":"string","nullable":true,"metadata":{}},{"name":"txnId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6015,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"5532233","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/trusted/function/load-trusted-sqledw-v2","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"980829582242690","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_project","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO SQLExecution: Received onJobStart for query with unregistered executionId 258 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:43.272Z","run":{"runId":"c5468f79-5bf8-4a4c-9636-8102ff8a4c31","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"path","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"partitionValues","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"size","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"modificationTime","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"dataChange","dataType":"boolean","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"stats","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6035,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"tags","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"deletionVector","dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SerializeFromObject","num-children":1,"serializer":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"path","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"path","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"partitionValues","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"partitionValues","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"size","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"size","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"modificationTime","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"modificationTime","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"dataChange","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"dataChange","dataType":"boolean","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"tags","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"tags","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"deletionVector","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]}},{"class":"org.apache.spark.sql.catalyst.expressions.CreateNamedStruct","num-children":10,"children":[0,1,2,3,4,5,6,7,8,9]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"storageType","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"storageType","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"pathOrInlineDv","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"pathOrInlineDv","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"offset","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.UnwrapOption","num-children":1,"dataType":"integer","child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"offset","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"sizeInBytes","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"sizeInBytes","dataType":"integer","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"cardinality","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"cardinality","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.MapElements","num-children":1,"func":null,"argumentClass":"scala.Tuple1","argumentSchema":{"type":"struct","fields":[{"name":"_1","type":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}}]},"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6026,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeserializeToObject","num-children":1,"deserializer":[{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":1,"cls":"scala.Tuple1","arguments":[0],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":8,"cls":"com.databricks.sql.transaction.tahoe.actions.AddFile","arguments":[0,1,2,3,4,5,6,7],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"path"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"partitionValues"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"size\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"size"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"modificationTime\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"modificationTime"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Boolean\", name: \"dataChange\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"dataChange"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":5,"name":"stats"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":6,"name":"tags"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":5,"cls":"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor","arguments":[0,1,2,3,4],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"storageType"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"pathOrInlineDv"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.WrapOption","num-children":1,"child":0,"optType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"offset"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Int\", name: \"sizeInBytes\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"sizeInBytes"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"cardinality\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"cardinality"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6025,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.IsNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"txn","dataType":{"type":"struct","fields":[{"name":"appId","type":"string","nullable":true,"metadata":{}},{"name":"version","type":"long","nullable":false,"metadata":{}},{"name":"lastUpdated","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6009,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"remove","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"deletionTimestamp","type":"long","nullable":true,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"extendedFileMetadata","type":"boolean","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6011,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"metaData","dataType":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"description","type":"string","nullable":true,"metadata":{}},{"name":"format","type":{"type":"struct","fields":[{"name":"provider","type":"string","nullable":true,"metadata":{}},{"name":"options","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"schemaString","type":"string","nullable":true,"metadata":{}},{"name":"partitionColumns","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"configuration","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"createdTime","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6012,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"protocol","dataType":{"type":"struct","fields":[{"name":"minReaderVersion","type":"integer","nullable":false,"metadata":{}},{"name":"minWriterVersion","type":"integer","nullable":false,"metadata":{}},{"name":"readerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"writerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6013,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"cdc","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6014,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"commitInfo","dataType":{"type":"struct","fields":[{"name":"version","type":"long","nullable":true,"metadata":{}},{"name":"timestamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"userId","type":"string","nullable":true,"metadata":{}},{"name":"userName","type":"string","nullable":true,"metadata":{}},{"name":"operation","type":"string","nullable":true,"metadata":{}},{"name":"operationParameters","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"job","type":{"type":"struct","fields":[{"name":"jobId","type":"string","nullable":true,"metadata":{}},{"name":"jobName","type":"string","nullable":true,"metadata":{}},{"name":"runId","type":"string","nullable":true,"metadata":{}},{"name":"jobOwnerId","type":"string","nullable":true,"metadata":{}},{"name":"triggerType","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"notebook","type":{"type":"struct","fields":[{"name":"notebookId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"clusterId","type":"string","nullable":true,"metadata":{}},{"name":"readVersion","type":"long","nullable":true,"metadata":{}},{"name":"isolationLevel","type":"string","nullable":true,"metadata":{}},{"name":"isBlindAppend","type":"boolean","nullable":true,"metadata":{}},{"name":"operationMetrics","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"userMetadata","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"engineInfo","type":"string","nullable":true,"metadata":{}},{"name":"txnId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6015,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_project","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:52:43 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:43 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:43.273Z","run":{"runId":"c5468f79-5bf8-4a4c-9636-8102ff8a4c31","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"path","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"partitionValues","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"size","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"modificationTime","dataType":"long","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"dataChange","dataType":"boolean","nullable":false,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"stats","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6035,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":"[__dataset_id, __col_position]"},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"string"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"tags","dataType":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"deletionVector","dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SerializeFromObject","num-children":1,"serializer":[[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"path","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6027,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"path","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"partitionValues","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6028,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-1},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-2},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"partitionValues","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"size","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6029,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"size","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"modificationTime","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6030,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"modificationTime","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"dataChange","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6031,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"dataChange","dataType":"boolean","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"tags","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6033,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.ExternalMapToCatalyst","num-children":5,"keyLoopVar":0,"keyConverter":1,"valueLoopVar":2,"valueConverter":3,"inputData":4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_key","dataType":"object","nullable":true,"id":-3},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"ExternalMapToCatalyst_value","dataType":"object","nullable":true,"id":-4},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"tags","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}],[{"class":"org.apache.spark.sql.catalyst.expressions.Alias","num-children":1,"child":0,"name":"deletionVector","exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6034,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[],"nonInheritableMetadataKeys":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]}},{"class":"org.apache.spark.sql.catalyst.expressions.CreateNamedStruct","num-children":10,"children":[0,1,2,3,4,5,6,7,8,9]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"storageType","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"storageType","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"pathOrInlineDv","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke","num-children":1,"staticObject":"org.apache.spark.unsafe.types.UTF8String","dataType":"string","functionName":"fromString","arguments":[0],"inputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"pathOrInlineDv","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"offset","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.UnwrapOption","num-children":1,"dataType":"integer","child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"offset","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"sizeInBytes","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"sizeInBytes","dataType":"integer","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"cardinality","dataType":"string"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"cardinality","dataType":"long","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"deletionVector","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":true,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.KnownNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[top level Product or row object]"},{"class":"org.apache.spark.sql.catalyst.expressions.BoundReference","num-children":0,"ordinal":0,"dataType":"object","nullable":true}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.MapElements","num-children":1,"func":null,"argumentClass":"scala.Tuple1","argumentSchema":{"type":"struct","fields":[{"name":"_1","type":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}}]},"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6026,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.DeserializeToObject","num-children":1,"deserializer":[{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":1,"cls":"scala.Tuple1","arguments":[0],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":8,"cls":"com.databricks.sql.transaction.tahoe.actions.AddFile","arguments":[0,1,2,3,4,5,6,7],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"path"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-5},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-6},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"partitionValues"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"size\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"size"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"modificationTime\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"modificationTime"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Boolean\", name: \"dataChange\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"dataChange"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":5,"name":"stats"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.CatalystToExternalMap","num-children":5,"keyLoopVar":0,"keyLambdaFunction":1,"valueLoopVar":2,"valueLambdaFunction":3,"inputData":4,"collClass":"scala.collection.immutable.Map"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_key","dataType":"string","nullable":false,"id":-7},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.objects.LambdaVariable","num-children":0,"name":"CatalystToExternalMap_value","dataType":"string","nullable":true,"id":-8},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":6,"name":"tags"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.If","num-children":3,"predicate":0,"trueValue":1,"falseValue":2},{"class":"org.apache.spark.sql.catalyst.expressions.IsNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":null,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.NewInstance","num-children":5,"cls":"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor","arguments":[0,1,2,3,4],"inputTypes":[],"propagateNull":false,"dataType":"object"},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":0,"name":"storageType"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.Invoke","num-children":1,"targetObject":0,"functionName":"toString","dataType":"object","arguments":[],"methodInputTypes":[],"propagateNull":true,"returnNullable":false,"isDeterministic":true},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":1,"name":"pathOrInlineDv"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.WrapOption","num-children":1,"child":0,"optType":"integer"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":2,"name":"offset"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Int\", name: \"sizeInBytes\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":3,"name":"sizeInBytes"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]},{"class":"org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull","num-children":1,"child":0,"walkedTypePath":"[- field (class: \"scala.Long\", name: \"cardinality\"), - field (class: \"com.databricks.sql.transaction.tahoe.actions.DeletionVectorDescriptor\", name: \"deletionVector\"), - root class: \"com.databricks.sql.transaction.tahoe.actions.AddFile\"]"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":4,"name":"cardinality"},{"class":"org.apache.spark.sql.catalyst.expressions.GetStructField","num-children":1,"child":0,"ordinal":7,"name":"deletionVector"},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"outputObjAttr":[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"obj","dataType":"object","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6025,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Filter","num-children":1,"condition":[{"class":"org.apache.spark.sql.catalyst.expressions.IsNotNull","num-children":1,"child":0},{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],"child":0},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"txn","dataType":{"type":"struct","fields":[{"name":"appId","type":"string","nullable":true,"metadata":{}},{"name":"version","type":"long","nullable":false,"metadata":{}},{"name":"lastUpdated","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6009,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"add","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"modificationTime","type":"long","nullable":false,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"stats","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6010,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"remove","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"deletionTimestamp","type":"long","nullable":true,"metadata":{}},{"name":"dataChange","type":"boolean","nullable":false,"metadata":{}},{"name":"extendedFileMetadata","type":"boolean","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"deletionVector","type":{"type":"struct","fields":[{"name":"storageType","type":"string","nullable":true,"metadata":{}},{"name":"pathOrInlineDv","type":"string","nullable":true,"metadata":{}},{"name":"offset","type":"integer","nullable":true,"metadata":{}},{"name":"sizeInBytes","type":"integer","nullable":false,"metadata":{}},{"name":"cardinality","type":"long","nullable":false,"metadata":{}}]},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6011,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"metaData","dataType":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"description","type":"string","nullable":true,"metadata":{}},{"name":"format","type":{"type":"struct","fields":[{"name":"provider","type":"string","nullable":true,"metadata":{}},{"name":"options","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"schemaString","type":"string","nullable":true,"metadata":{}},{"name":"partitionColumns","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"configuration","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"createdTime","type":"long","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6012,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"protocol","dataType":{"type":"struct","fields":[{"name":"minReaderVersion","type":"integer","nullable":false,"metadata":{}},{"name":"minWriterVersion","type":"integer","nullable":false,"metadata":{}},{"name":"readerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}},{"name":"writerFeatures","type":{"type":"array","elementType":"string","containsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6013,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"cdc","dataType":{"type":"struct","fields":[{"name":"path","type":"string","nullable":true,"metadata":{}},{"name":"partitionValues","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"size","type":"long","nullable":false,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6014,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"commitInfo","dataType":{"type":"struct","fields":[{"name":"version","type":"long","nullable":true,"metadata":{}},{"name":"timestamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"userId","type":"string","nullable":true,"metadata":{}},{"name":"userName","type":"string","nullable":true,"metadata":{}},{"name":"operation","type":"string","nullable":true,"metadata":{}},{"name":"operationParameters","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"job","type":{"type":"struct","fields":[{"name":"jobId","type":"string","nullable":true,"metadata":{}},{"name":"jobName","type":"string","nullable":true,"metadata":{}},{"name":"runId","type":"string","nullable":true,"metadata":{}},{"name":"jobOwnerId","type":"string","nullable":true,"metadata":{}},{"name":"triggerType","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"notebook","type":{"type":"struct","fields":[{"name":"notebookId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{}},{"name":"clusterId","type":"string","nullable":true,"metadata":{}},{"name":"readVersion","type":"long","nullable":true,"metadata":{}},{"name":"isolationLevel","type":"string","nullable":true,"metadata":{}},{"name":"isBlindAppend","type":"boolean","nullable":true,"metadata":{}},{"name":"operationMetrics","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"userMetadata","type":"string","nullable":true,"metadata":{}},{"name":"tags","type":{"type":"map","keyType":"string","valueType":"string","valueContainsNull":true},"nullable":true,"metadata":{}},{"name":"engineInfo","type":"string","nullable":true,"metadata":{}},{"name":"txnId","type":"string","nullable":true,"metadata":{}}]},"nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6015,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_project","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:43 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json using temp file dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp 23/06/28 07:52:43 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp] Closing stream; size: 1792 23/06/28 07:52:43 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp] Upload complete; size: 1792 23/06/28 07:52:43 INFO NativeAzureFileSystem: FS_OP_RENAME SRC[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp] DST[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json] Starting rename. Issuing rename operation. 23/06/28 07:52:43 INFO NativeAzureFileSystem: FS_OP_RENAME SRC[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp] DST[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json] Rename successful. 23/06/28 07:52:43 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.json.7b9bfa3c-68b0-431c-b600-7e8db38dda50.tmp to dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json 23/06/28 07:52:43 INFO OptimisticTransaction: [tableId=e48355b8,txnId=46fb154c] Incremental commit: starting with snapshot version -1 23/06/28 07:52:43 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:52:43 INFO CodeGenerator: Code generated in 33.325218 ms 23/06/28 07:52:43 INFO SparkContext: Starting job: collect at Checksum.scala:423 23/06/28 07:52:43 INFO DAGScheduler: Job 84 finished: collect at Checksum.scala:423, took 0.000181 s 23/06/28 07:52:43 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:52:43 INFO SQLExecution: Received onJobStart for query with unregistered executionId 259 23/06/28 07:52:43 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:52:43 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:52:43 INFO DeltaLog: Loading version 0. 23/06/28 07:52:43 INFO SnapshotEdge: [tableId=e48355b8-0ee0-4734-8f1b-17582bd0a130] Created snapshot SnapshotEdge(path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log, version=0, metadata=Metadata(0a243321-de86-4bc0-9520-53c494e03423,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},ArrayBuffer(),Map(),Some(1687938757305)), logSegment=LogSegment(dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log,0,List(FileStatus{path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json; isDirectory=false; length=1792; replication=1; blocksize=536870912; modification_time=1687938763000; access_time=0; owner=; group=; permission=rwx-wx-wx; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}),List(),None,1687938763000), checksumOpt=Some(VersionChecksum(Some(46fb154c-4708-41c6-9c45-1fca3de21dc3),7183,1,None,None,1,1,Some(Stream()),Metadata(0a243321-de86-4bc0-9520-53c494e03423,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},ArrayBuffer(),Map(),Some(1687938757305)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@1b3d9a52,[J@7b4f945e)),None,Some(Stream(AddFile(part-00191-6ba3ae06-38f6-4096-8a9a-fe2edda2bc88-c000.snappy.parquet,Map(),7183,1687938762000,false,{"numRecords":1,"minValues":{"id":"1","sql":"WITH tbl_src AS (\r\nSELECT * FROM"},"maxValues":{"id":"1","sql":"WITH tbl_src AS (\r\nSELECT * FROM�"},"nullCount":{"id":0,"sql":0}},Map(INSERTION_TIME -> 1687938762000000, MIN_INSERTION_TIME -> 1687938762000000, MAX_INSERTION_TIME -> 1687938762000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:52:43 INFO MemoryStore: Block broadcast_130 stored as values in memory (estimated size 508.7 KiB, free 750.6 GiB) 23/06/28 07:52:43 INFO MemoryStore: Block broadcast_130_piece0 stored as bytes in memory (estimated size 13.7 KiB, free 750.6 GiB) 23/06/28 07:52:43 INFO BlockManagerInfo: Added broadcast_130_piece0 in memory on 10.60.24.15:40507 (size: 13.7 KiB, free: 750.7 GiB) 23/06/28 07:52:43 INFO SparkContext: Created broadcast 130 from broadcast at DeltaLog.scala:634 23/06/28 07:52:43 INFO DeltaLogFileIndex: Created DeltaLogFileIndex(JSON, numFilesInSegment: 1, totalFileSize: 1792) 23/06/28 07:52:43 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:52:43 INFO CodeGenerator: Code generated in 6.413259 ms 23/06/28 07:52:43 INFO CodeGenerator: Code generated in 15.244793 ms 23/06/28 07:52:43 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:52:44 INFO CodeGenerator: Code generated in 20.812333 ms 23/06/28 07:52:44 INFO DAGScheduler: Registering RDD 424 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 48 23/06/28 07:52:44 INFO DAGScheduler: Got map stage job 85 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 4 output partitions 23/06/28 07:52:44 INFO DAGScheduler: Final stage: ShuffleMapStage 197 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:52:44 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:52:44 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:44 INFO DAGScheduler: Submitting ShuffleMapStage 197 (MapPartitionsRDD[424] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:52:44 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:44 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:44 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:44 INFO DAGScheduler: Submitting 4 missing tasks from ShuffleMapStage 197 (MapPartitionsRDD[424] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Adding task set 197.0 with 4 tasks resource profile 0 23/06/28 07:52:44 INFO FairSchedulableBuilder: Added task set TaskSet_197.0 tasks to pool 6803957155303634069 23/06/28 07:52:44 INFO TaskSetManager: Starting task 0.0 in stage 197.0 (TID 1568) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO TaskSetManager: Starting task 1.0 in stage 197.0 (TID 1569) (10.60.24.10, executor 0, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO TaskSetManager: Starting task 2.0 in stage 197.0 (TID 1570) (10.60.24.13, executor 1, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO TaskSetManager: Starting task 3.0 in stage 197.0 (TID 1571) (10.60.24.10, executor 0, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_131 stored as values in memory (estimated size 133.0 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_131_piece0 stored as bytes in memory (estimated size 23.0 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_131_piece0 in memory on 10.60.24.15:40507 (size: 23.0 KiB, free: 750.7 GiB) 23/06/28 07:52:44 INFO SparkContext: Created broadcast 131 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_131_piece0 in memory on 10.60.24.10:41041 (size: 23.0 KiB, free: 487.7 GiB) 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_131_piece0 in memory on 10.60.24.13:44035 (size: 23.0 KiB, free: 487.7 GiB) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 1.0 in stage 197.0 (TID 1569) in 15 ms on 10.60.24.10 (executor 0) (1/4) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 2.0 in stage 197.0 (TID 1570) in 15 ms on 10.60.24.13 (executor 1) (2/4) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 3.0 in stage 197.0 (TID 1571) in 16 ms on 10.60.24.10 (executor 0) (3/4) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 0.0 in stage 197.0 (TID 1568) in 16 ms on 10.60.24.13 (executor 1) (4/4) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Removed TaskSet 197.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:44 INFO DAGScheduler: ShuffleMapStage 197 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.018 s 23/06/28 07:52:44 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:52:44 INFO DAGScheduler: running: Set() 23/06/28 07:52:44 INFO DAGScheduler: waiting: Set() 23/06/28 07:52:44 INFO DAGScheduler: failed: Set() 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:52:44 INFO CodeGenerator: Code generated in 34.615785 ms 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:52:44 INFO SparkContext: Starting job: collect at Checksum.scala:561 23/06/28 07:52:44 INFO DAGScheduler: Got job 86 (collect at Checksum.scala:561) with 1 output partitions 23/06/28 07:52:44 INFO DAGScheduler: Final stage: ResultStage 199 (collect at Checksum.scala:561) 23/06/28 07:52:44 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 198) 23/06/28 07:52:44 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:44 INFO DAGScheduler: Submitting ResultStage 199 (MapPartitionsRDD[435] at collect at Checksum.scala:561), which has no missing parents 23/06/28 07:52:44 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:44 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:44 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:44 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 199 (MapPartitionsRDD[435] at collect at Checksum.scala:561) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Adding task set 199.0 with 1 tasks resource profile 0 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:52:44 INFO FairSchedulableBuilder: Added task set TaskSet_199.0 tasks to pool 6803957155303634069 23/06/28 07:52:44 INFO TaskSetManager: Starting task 0.0 in stage 199.0 (TID 1572) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_132 stored as values in memory (estimated size 454.3 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_132_piece0 stored as bytes in memory (estimated size 106.9 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_132_piece0 in memory on 10.60.24.15:40507 (size: 106.9 KiB, free: 750.7 GiB) 23/06/28 07:52:44 INFO SparkContext: Created broadcast 132 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_132_piece0 in memory on 10.60.24.10:41041 (size: 106.9 KiB, free: 487.7 GiB) 23/06/28 07:52:44 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 48 to 10.60.24.10:44288 23/06/28 07:52:44 INFO BlockManagerInfo: Added rdd_431_0 in memory on 10.60.24.10:41041 (size: 857.0 B, free: 487.7 GiB) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 0.0 in stage 199.0 (TID 1572) in 97 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Removed TaskSet 199.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:44 INFO DAGScheduler: ResultStage 199 (collect at Checksum.scala:561) finished in 0.113 s 23/06/28 07:52:44 INFO DAGScheduler: Job 86 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:52:44 INFO TaskSchedulerImpl: Killing all running tasks in stage 199: Stage finished 23/06/28 07:52:44 INFO DAGScheduler: Job 86 finished: collect at Checksum.scala:561, took 0.115213 s 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:52:44 INFO CodeGenerator: Code generated in 24.485535 ms 23/06/28 07:52:44 INFO DAGScheduler: Registering RDD 438 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 49 23/06/28 07:52:44 INFO DAGScheduler: Got map stage job 87 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:52:44 INFO DAGScheduler: Final stage: ShuffleMapStage 201 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:52:44 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 200) 23/06/28 07:52:44 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:44 INFO DAGScheduler: Submitting ShuffleMapStage 201 (MapPartitionsRDD[438] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:52:44 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:44 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:44 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:44 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 201 (MapPartitionsRDD[438] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Adding task set 201.0 with 1 tasks resource profile 0 23/06/28 07:52:44 INFO FairSchedulableBuilder: Added task set TaskSet_201.0 tasks to pool 6803957155303634069 23/06/28 07:52:44 INFO TaskSetManager: Starting task 0.0 in stage 201.0 (TID 1573) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_133 stored as values in memory (estimated size 424.6 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_133_piece0 stored as bytes in memory (estimated size 101.4 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_133_piece0 in memory on 10.60.24.15:40507 (size: 101.4 KiB, free: 750.7 GiB) 23/06/28 07:52:44 INFO SparkContext: Created broadcast 133 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_133_piece0 in memory on 10.60.24.10:41041 (size: 101.4 KiB, free: 487.7 GiB) 23/06/28 07:52:44 INFO TaskSetManager: Finished task 0.0 in stage 201.0 (TID 1573) in 32 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Removed TaskSet 201.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:44 INFO DAGScheduler: ShuffleMapStage 201 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 0.041 s 23/06/28 07:52:44 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:52:44 INFO DAGScheduler: running: Set() 23/06/28 07:52:44 INFO DAGScheduler: waiting: Set() 23/06/28 07:52:44 INFO DAGScheduler: failed: Set() 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:52:44 INFO SparkContext: Starting job: first at Snapshot.scala:271 23/06/28 07:52:44 INFO DAGScheduler: Got job 88 (first at Snapshot.scala:271) with 1 output partitions 23/06/28 07:52:44 INFO DAGScheduler: Final stage: ResultStage 204 (first at Snapshot.scala:271) 23/06/28 07:52:44 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 203) 23/06/28 07:52:44 INFO DAGScheduler: Missing parents: List() 23/06/28 07:52:44 INFO DAGScheduler: Submitting ResultStage 204 (MapPartitionsRDD[440] at first at Snapshot.scala:271), which has no missing parents 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:52:44 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:52:44 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:52:44 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:52:44 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 204 (MapPartitionsRDD[440] at first at Snapshot.scala:271) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Adding task set 204.0 with 1 tasks resource profile 0 23/06/28 07:52:44 INFO FairSchedulableBuilder: Added task set TaskSet_204.0 tasks to pool 6803957155303634069 23/06/28 07:52:44 INFO TaskSetManager: Starting task 0.0 in stage 204.0 (TID 1574) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_134 stored as values in memory (estimated size 372.1 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO MemoryStore: Block broadcast_134_piece0 stored as bytes in memory (estimated size 92.5 KiB, free 750.6 GiB) 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_134_piece0 in memory on 10.60.24.15:40507 (size: 92.5 KiB, free: 750.7 GiB) 23/06/28 07:52:44 INFO SparkContext: Created broadcast 134 from broadcast at TaskSetManager.scala:627 23/06/28 07:52:44 INFO BlockManagerInfo: Added broadcast_134_piece0 in memory on 10.60.24.10:41041 (size: 92.5 KiB, free: 487.7 GiB) 23/06/28 07:52:44 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 49 to 10.60.24.10:44288 23/06/28 07:52:44 INFO TaskSetManager: Finished task 0.0 in stage 204.0 (TID 1574) in 31 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:52:44 INFO TaskSchedulerImpl: Removed TaskSet 204.0, whose tasks have all completed, from pool 6803957155303634069 23/06/28 07:52:44 INFO DAGScheduler: ResultStage 204 (first at Snapshot.scala:271) finished in 0.041 s 23/06/28 07:52:44 INFO DAGScheduler: Job 88 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:52:44 INFO TaskSchedulerImpl: Killing all running tasks in stage 204: Stage finished 23/06/28 07:52:44 INFO DAGScheduler: Job 88 finished: first at Snapshot.scala:271, took 0.042476 s 23/06/28 07:52:44 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:52:44 INFO DeltaLog: Updated snapshot to SnapshotEdge(path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log, version=0, metadata=Metadata(0a243321-de86-4bc0-9520-53c494e03423,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},ArrayBuffer(),Map(),Some(1687938757305)), logSegment=LogSegment(dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log,0,List(FileStatus{path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.json; isDirectory=false; length=1792; replication=1; blocksize=536870912; modification_time=1687938763000; access_time=0; owner=; group=; permission=rwx-wx-wx; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}),List(),None,1687938763000), checksumOpt=Some(VersionChecksum(Some(46fb154c-4708-41c6-9c45-1fca3de21dc3),7183,1,None,None,1,1,Some(Stream()),Metadata(0a243321-de86-4bc0-9520-53c494e03423,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},ArrayBuffer(),Map(),Some(1687938757305)),Protocol(1,2),Some(FileSizeHistogram(Vector(0, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 12582912, 16777216, 20971520, 25165824, 29360128, 33554432, 37748736, 41943040, 50331648, 58720256, 67108864, 75497472, 83886080, 92274688, 100663296, 109051904, 117440512, 125829120, 130023424, 134217728, 138412032, 142606336, 146800640, 150994944, 167772160, 184549376, 201326592, 218103808, 234881024, 251658240, 268435456, 285212672, 301989888, 318767104, 335544320, 352321536, 369098752, 385875968, 402653184, 419430400, 436207616, 452984832, 469762048, 486539264, 503316480, 520093696, 536870912, 553648128, 570425344, 587202560, 603979776, 671088640, 738197504, 805306368, 872415232, 939524096, 1006632960, 1073741824, 1140850688, 1207959552, 1275068416, 1342177280, 1409286144, 1476395008, 1610612736, 1744830464, 1879048192, 2013265920, 2147483648, 2415919104, 2684354560, 2952790016, 3221225472, 3489660928, 3758096384, 4026531840, 4294967296, 8589934592, 17179869184, 34359738368, 68719476736, 137438953472, 274877906944),[J@1b3d9a52,[J@7b4f945e)),None,Some(Stream(AddFile(part-00191-6ba3ae06-38f6-4096-8a9a-fe2edda2bc88-c000.snappy.parquet,Map(),7183,1687938762000,false,{"numRecords":1,"minValues":{"id":"1","sql":"WITH tbl_src AS (\r\nSELECT * FROM"},"maxValues":{"id":"1","sql":"WITH tbl_src AS (\r\nSELECT * FROM�"},"nullCount":{"id":0,"sql":0}},Map(INSERTION_TIME -> 1687938762000000, MIN_INSERTION_TIME -> 1687938762000000, MAX_INSERTION_TIME -> 1687938762000000, OPTIMIZE_TARGET_SIZE -> 268435456),null)))))) 23/06/28 07:52:44 INFO FileSizeAutoTuner: File size tuning result: {"tuningType":"autoTuned","tunedConfs":{"spark.databricks.delta.optimize.minFileSize":"268435456","spark.databricks.delta.autoCompact.maxFileSize":"16777216","spark.databricks.delta.optimize.maxFileSize":"268435456","spark.databricks.delta.autoCompact.minFileSize":"8388608"}} 23/06/28 07:52:44 INFO OptimisticTransaction: [tableId=e48355b8,txnId=46fb154c] Committed delta #0 to dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log 23/06/28 07:52:44 INFO ChecksumHook: Writing checksum file for table path dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log version 0 23/06/28 07:52:44 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp] Creating output stream; permission: { masked: rw-r--r--, unmasked: rw-rw-rw- }, overwrite: false, bufferSize: 65536 23/06/28 07:52:44 INFO RetryTolerableRenameFSDataOutputStream: Writing atomically to dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.crc using temp file dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp 23/06/28 07:52:44 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp] Closing stream; size: 2502 23/06/28 07:52:44 INFO NativeAzureFileSystem: FS_OP_CREATE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp] Upload complete; size: 2502 23/06/28 07:52:44 INFO NativeAzureFileSystem: FS_OP_RENAME SRC[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp] DST[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.crc] Starting rename. Issuing rename operation. 23/06/28 07:52:44 INFO NativeAzureFileSystem: FS_OP_RENAME SRC[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp] DST[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.crc] Rename successful. 23/06/28 07:52:44 INFO RetryTolerableRenameFSDataOutputStream: Renamed temp file dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/__tmp_path_dir/.00000000000000000000.crc.c01cb90b-520e-4366-9897-372cff6b92bd.tmp to dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge/_delta_log/00000000000000000000.crc 23/06/28 07:52:44 INFO CreateDeltaTableCommand: Table is path-based table: false. Update catalog with mode: CreateOrReplace 23/06/28 07:52:44 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:44 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:44 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:44 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 INFO HiveMetaStore: 12: get_database: default 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:45 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 WARN SecuredHiveExternalCatalog: Couldn't find corresponding Hive SerDe for data source provider delta. Persisting data source table `spark_catalog`.`default`.`lms_lmsuu_mdl_groups_members_sql_merge` into Hive metastore in Spark SQL specific format, which is NOT compatible with Hive. 23/06/28 07:52:45 INFO HiveMetaStore: 12: create_table: Table(tableName:lms_lmsuu_mdl_groups_members_sql_merge, dbName:default, owner:root, createTime:1687938756, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:col, type:array, comment:from deserializer)], location:null, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge, serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{})), partitionKeys:[], parameters:{delta.minReaderVersion=1, delta.minWriterVersion=2, spark.sql.sources.schema={"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]}, spark.sql.partitionProvider=catalog, spark.sql.sources.provider=delta, delta.lastCommitTimestamp=1687938763000, spark.sql.create.version=3.3.1, delta.lastUpdateVersion=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=create_table: Table(tableName:lms_lmsuu_mdl_groups_members_sql_merge, dbName:default, owner:root, createTime:1687938756, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:col, type:array, comment:from deserializer)], location:null, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{path=dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge, serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{})), partitionKeys:[], parameters:{delta.minReaderVersion=1, delta.minWriterVersion=2, spark.sql.sources.schema={"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]}, spark.sql.partitionProvider=catalog, spark.sql.sources.provider=delta, delta.lastCommitTimestamp=1687938763000, spark.sql.create.version=3.3.1, delta.lastUpdateVersion=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) 23/06/28 07:52:45 INFO HiveMetaStore: 12: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 INFO ClusterLoadMonitor: Removed query with execution ID:256. Current active queries:0 23/06/28 07:52:45 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:45 INFO HiveMetaStore: 2: get_database: default 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:45 INFO ReliableJobsSessionClient: Created ReliableJobsSessionClient with timeout of 0 seconds 23/06/28 07:52:45 INFO HiveMetaStore: 2: get_database: default 23/06/28 07:52:45 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:52:45 INFO AbstractDatabricksHandler: dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:52:45 WARN InputFieldsCollector: Could not extract dataset identifier from org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier 23/06/28 07:52:45 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:45.585Z","run":{"runId":"0eb11396-af76-4bf3-a6fb-451e49ce1273","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.ReplaceTableAsSelect","num-children":2,"name":0,"partitioning":[],"query":1,"tableSpec":null,"writeOptions":null,"orCreate":true},{"class":"org.apache.spark.sql.catalyst.analysis.ResolvedIdentifier","num-children":0,"catalog":null,"identifier":null},{"class":"org.apache.spark.sql.execution.LogicalRDD","num-children":0,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5936,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":5937,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"rdd":null,"outputPartitioning":{"product-class":"org.apache.spark.sql.catalyst.plans.physical.UnknownPartitioning","numPartitions":0},"outputOrdering":[],"isStreaming":false,"session":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_atomic_replace_table_as_select_warehouse_lms_lmsuu_mdl_groups_members_sql_merge","facets":{}},"inputs":[],"outputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"storage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/StorageDatasetFacet.json#/$defs/StorageDatasetFacet","storageLayer":"unity","fileFormat":"parquet"},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"default.LMS_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:46 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:52:48 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:52:48 WARN SQLConf: The SQL config 'spark.sql.hive.convertCTAS' has been deprecated in Spark v3.1 and may be removed in the future. Set 'spark.sql.legacy.createHiveTableByDefault' to false instead. 23/06/28 07:52:48 WARN SimpleTableFunctionRegistry: The function list_secrets replaced a previously registered function. 23/06/28 07:52:48 WARN SimpleTableFunctionRegistry: The function read_files replaced a previously registered function. 23/06/28 07:52:48 INFO DriverCorral: Starting scala repl ReplId-37b9f-dc475-6cd58-8 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:263. Current active queries:1 23/06/28 07:52:48 INFO LogicalPlanStats: Setting LogicalPlanStats visitor to com.databricks.sql.optimizer.statsEstimation.DatabricksLogicalPlanStatsVisitor$ 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:263. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:264. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:264. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:265. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:265. Current active queries:0 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:48.758Z","run":{"runId":"a1bbcc41-0fac-47bf-a904-11372553b529","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:266. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.758Z","run":{"runId":"a1bbcc41-0fac-47bf-a904-11372553b529","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:266. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:267. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:267. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:268. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.763Z","run":{"runId":"d665d21d-5641-481a-b54f-7efba7924b38","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:268. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:269. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:269. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:270. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.766Z","run":{"runId":"f7a7ee35-0235-435a-a21e-f44f675b154c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:270. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:271. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:271. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:272. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.77Z","run":{"runId":"c0879047-d367-449b-9f60-1be123b4581c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:272. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:273. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:273. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:274. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.774Z","run":{"runId":"bcbe04fb-a66b-40e5-ac0f-6f9477bd9825","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:274. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:275. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:275. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:276. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.778Z","run":{"runId":"01498ac9-7118-46ef-8c78-b04c39d695ff","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:276. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:277. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:277. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:278. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.782Z","run":{"runId":"c02a2dd9-1805-4b96-9e8a-3f9a8a115d2b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:278. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:279. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:279. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:280. Current active queries:1 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:280. Current active queries:0 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.786Z","run":{"runId":"71c95ac0-f4db-4e6f-aa57-c1411448e475","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:281. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:281. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:282. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:282. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.789Z","run":{"runId":"4a215b07-cd63-4f7c-9ea3-be814f7c9938","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:283. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:283. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:284. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:284. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.793Z","run":{"runId":"0e1e9c30-2104-48e1-b813-13d3abc63efb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:285. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:285. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:286. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:286. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.796Z","run":{"runId":"c8336ec3-b1dc-42d8-8593-a778ae87c80c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:287. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:287. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:288. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:288. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.8Z","run":{"runId":"63fde9ad-c957-4a3d-ac4c-d7995467734c","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:289. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:289. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:290. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:290. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.804Z","run":{"runId":"940e9798-d4d0-4c6a-b202-b93e217d334d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:291. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:291. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:292. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:292. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:293. Current active queries:1 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.807Z","run":{"runId":"19090801-8d49-4994-bb3d-0bbc20115d58","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:293. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:294. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:294. Current active queries:0 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:295. Current active queries:1 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.811Z","run":{"runId":"7266518d-0ba7-43aa-a752-856f8fd9a2bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:295. Current active queries:0 23/06/28 07:52:48 INFO ClusterLoadMonitor: Added query with execution ID:296. Current active queries:1 23/06/28 07:52:48 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:48 INFO ClusterLoadMonitor: Removed query with execution ID:296. Current active queries:0 23/06/28 07:52:48 INFO ScalaDriverWrapper: setupRepl:ReplId-37b9f-dc475-6cd58-8: finished to load 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.814Z","run":{"runId":"93b7ee0b-bfed-49ca-9676-ce142cf6ed37","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.818Z","run":{"runId":"a4c51bed-4444-4407-8984-18e3910e29d3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.821Z","run":{"runId":"68893c4f-8186-44c0-acda-cab87834f193","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.825Z","run":{"runId":"e9fa9176-925f-4bf7-8431-78fdc03fb9c0","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.828Z","run":{"runId":"5c03401c-c4b1-4a5e-9d69-ab9aa8767a82","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.832Z","run":{"runId":"b60c429c-db03-410b-b984-e846e5d55e35","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.836Z","run":{"runId":"72ff8960-a852-422d-8572-c185507fb8b6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.839Z","run":{"runId":"cfae9cf3-c8c7-4a79-bd62-5d68689a873b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.843Z","run":{"runId":"a3994847-54f9-4adf-944f-72ba0f0a24e6","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.846Z","run":{"runId":"58bbed4e-3568-4e10-b8e1-fb1063404e99","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.85Z","run":{"runId":"2002337c-9aae-4b74-a168-8999ffedc4f1","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.853Z","run":{"runId":"d3be3f68-a975-4999-9c42-4cc5ef309210","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.857Z","run":{"runId":"b9b6d37c-321b-48fb-ac68-6dc3e72bf5bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.86Z","run":{"runId":"82855b2e-3189-4147-8a8b-77d46319dc9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.864Z","run":{"runId":"3015c85d-4820-490e-8367-2b211b483227","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.867Z","run":{"runId":"7e05a966-9dab-4112-8aec-a7b075975d9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:48 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:48 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.871Z","run":{"runId":"807a95d0-1db6-459b-981d-313af4b93ce8","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.874Z","run":{"runId":"43db2ee0-9887-4f95-ad33-94cdc9caf981","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:48.878Z","run":{"runId":"e28c7773-ea66-4b83-972e-14b9ee45b3bb","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO DriverCorral: Starting python repl ReplId-211eb-e6200-f0983-4 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:297. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar at spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:297. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:298. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar at spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:298. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:299. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar at spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:299. Current active queries:0 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:52:49.244Z","run":{"runId":"15904897-46e5-4eae-96b9-3ea2af662eb3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.245Z","run":{"runId":"15904897-46e5-4eae-96b9-3ea2af662eb3","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:300. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar at spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:300. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:301. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar at spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:301. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.25Z","run":{"runId":"7ba87da6-4146-4578-a228-78ae5c861540","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:302. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar at spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:302. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:303. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar at spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:303. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.254Z","run":{"runId":"59c96abf-35db-4839-b1d6-2c893a71c6af","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:304. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar at spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:304. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:305. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar at spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:305. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.258Z","run":{"runId":"51ceaca4-fc69-498b-a880-65217b9aaa9a","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:306. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar at spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:306. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:307. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar at spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:307. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:308. Current active queries:1 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.262Z","run":{"runId":"3f34b363-50bc-4c8e-b6af-e08b34e9664f","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar at spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:308. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:309. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar at spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:309. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.266Z","run":{"runId":"3a804005-72a6-4111-b4a4-c44bc358df79","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:310. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar at spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:310. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:311. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar at spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:311. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.27Z","run":{"runId":"94a79879-f300-4706-981f-dbd3c87d0825","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:312. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar at spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:312. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:313. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar at spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:313. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.274Z","run":{"runId":"79eed10a-7a3c-479c-bec3-75e9610c70ae","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:314. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar at spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:314. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:315. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar at spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:315. Current active queries:0 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.278Z","run":{"runId":"266605cb-704a-4317-b25a-91186204b32b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:316. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar at spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:316. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:317. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar at spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:317. Current active queries:0 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.282Z","run":{"runId":"24b56d09-e65b-4236-9676-0f35656944c0","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:318. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar at spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:318. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:319. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar at spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:319. Current active queries:0 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.285Z","run":{"runId":"d544c143-53c5-4a8b-ab1b-207082f4d100","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:320. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar at spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:320. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:321. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.289Z","run":{"runId":"4c4af7c7-62c9-4f19-8246-22a3d6bca364","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar at spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:321. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:322. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar at spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:322. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:323. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.293Z","run":{"runId":"80b864c5-e2ba-4096-a6fc-6597676e3fe5","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar at spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:323. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:324. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar at spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:324. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:325. Current active queries:1 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.297Z","run":{"runId":"7e191527-2f1b-49d4-bd46-87db12a938e7","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar at spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:325. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:326. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar at spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:326. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.3Z","run":{"runId":"f981d499-91d3-4586-bd99-e753f810d115","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:327. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar at spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:327. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:328. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar at spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:328. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.305Z","run":{"runId":"bff1d9c2-2712-4269-908b-4779c4619b05","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:329. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar at spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:329. Current active queries:0 23/06/28 07:52:49 INFO ClusterLoadMonitor: Added query with execution ID:330. Current active queries:1 23/06/28 07:52:49 INFO SparkContext: The JAR file:/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar at spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar has been added already. Overwriting of added jar is not supported in the current version. 23/06/28 07:52:49 INFO ClusterLoadMonitor: Removed query with execution ID:330. Current active queries:0 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.308Z","run":{"runId":"1d04e7a0-2631-4011-92e0-ea21dcde2b37","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO JupyterDriverLocal: Starting gateway server for repl ReplId-211eb-e6200-f0983-4 23/06/28 07:52:49 INFO PythonPy4JUtil: Using pinned thread mode in Py4J 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.312Z","run":{"runId":"a46b8a77-99f8-4940-aeda-ffec92af6b85","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.316Z","run":{"runId":"93e39da2-8a88-4cc6-bc86-a675e8b2b70b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.32Z","run":{"runId":"0c07c802-545f-4b48-ac71-9fd9a8b9dbcd","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.324Z","run":{"runId":"ca752f86-6898-4a38-83fb-27e64ff10273","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.328Z","run":{"runId":"123a540a-bee7-4760-9370-e8757c6649a0","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.331Z","run":{"runId":"b3c32d71-288f-441a-90e2-7f66cd694815","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.335Z","run":{"runId":"1196a657-ad55-47f2-a0a9-34efd0ff9c45","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.339Z","run":{"runId":"6db60aab-f68d-4f22-9a82-960d47ed97b2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.343Z","run":{"runId":"231f1a90-7af8-4331-bdbe-540e259e1943","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.347Z","run":{"runId":"b8fc9fd0-6863-4ab9-9f2a-1c3934c0efbf","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.351Z","run":{"runId":"5caebfaa-6e9b-457e-88e0-2e18aff56476","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.354Z","run":{"runId":"243e4855-fa94-44a4-909b-380fabe58f57","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.359Z","run":{"runId":"4f898b8b-c3d1-4b5d-8e9d-7eb135eab605","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.363Z","run":{"runId":"4149e778-79da-495b-b8a9-2116772aad3b","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.367Z","run":{"runId":"14c448c5-b885-4c4f-b8d4-5ec8ed51647d","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.37Z","run":{"runId":"2246a4ae-5d9a-4db1-9ace-3b0fb174eb43","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:52:49 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:52:49.374Z","run":{"runId":"5a4ed0e4-c82d-4bf5-95a4-adc240e9b1f1","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.AddJarsCommand","num-children":0,"paths":"[/local_disk0/tmp/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar]"}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_add_jars_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:52:49 INFO VirtualenvCloneHelper: Creating notebook-scoped virtualenv for 7ae9ba6b-0522-437e-9465-929e20755c2d 23/06/28 07:52:49 INFO Utils: resolved command to be run: List(virtualenv, /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d, -p, /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, --no-download, --no-setuptools, --no-wheel) 23/06/28 07:52:49 INFO PythonEnvCloneHelper$: Created python virtualenv: /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d 23/06/28 07:52:49 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/python, -c, import sys; dirs=[p for p in sys.path if 'package' in p]; print(' '.join(dirs))) 23/06/28 07:52:49 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:52:49 INFO Utils: resolved command to be run: List(/local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d/bin/python, -c, from distutils.sysconfig import get_python_lib; print(get_python_lib())) 23/06/28 07:52:49 INFO PythonEnvCloneHelper$: Created sites.pth at /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d/lib/python3.9/site-packages/sites.pth 23/06/28 07:52:49 INFO NotebookScopedPythonEnvManager: Time spent to start virtualenv /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d is 205(ms) 23/06/28 07:52:49 INFO NotebookScopedPythonEnvManager: Registered /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d/lib/python3.9/site-packages with the WatchService sun.nio.fs.LinuxWatchService$LinuxWatchKey@1cfa9052 23/06/28 07:52:49 INFO IpykernelUtils$: Python process builder: [bash, /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d/python_start_7ae9ba6b-0522-437e-9465-929e20755c2d.sh, /databricks/spark/python/pyspark/wrapped_python.py, root, /local_disk0/.ephemeral_nfs/envs/pythonEnv-7ae9ba6b-0522-437e-9465-929e20755c2d/bin/python, /databricks/python_shell/scripts/db_ipykernel_launcher.py, -f, /databricks/kernel-connections/66d2662e511f33b62fd1f634560a812a4f897e7d523b8525d3a429dd40c3d62e.json] 23/06/28 07:52:49 INFO IpykernelUtils$: Cgroup isolation disabled, not placing python process in repl cgroup 23/06/28 07:52:51 INFO PythonDriverWrapper: setupRepl:ReplId-211eb-e6200-f0983-4: finished to load 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5713091679139387796_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5713091679139387796_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInRead' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInRead' instead. 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5713091679139387796_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8888146966390938293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8888146966390938293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 WARN SQLConf: The SQL config 'spark.sql.legacy.parquet.int96RebaseModeInWrite' has been deprecated in Spark v3.2 and may be removed in the future. Use 'spark.sql.parquet.int96RebaseModeInWrite' instead. 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8888146966390938293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6495072273175140215_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6495072273175140215_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6495072273175140215_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7911159048277729851_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7911159048277729851_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7911159048277729851_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7585739273496339042_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:51 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7585739273496339042_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:52 INFO DriverILoop: Set class prefix to: $line50be7afa7fcf457089dabef28efcebee 23/06/28 07:52:52 INFO DriverILoop: set ContextClassLoader 23/06/28 07:52:52 INFO DriverILoop: initialized intp 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7585739273496339042_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6507309663443579486_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6507309663443579486_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6507309663443579486_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8323558489890206473_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8323558489890206473_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8323558489890206473_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5589018390575261938_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5589018390575261938_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5589018390575261938_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6795810628280080235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6795810628280080235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6795810628280080235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7011117764275751085_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7011117764275751085_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7011117764275751085_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4867376821892200034_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:53 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4867376821892200034_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4867376821892200034_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7561051923659991431_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7561051923659991431_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7561051923659991431_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5106890152166825172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5106890152166825172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5106890152166825172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5071236899714059252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5071236899714059252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5071236899714059252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8186395924090113144_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8186395924090113144_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8186395924090113144_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6071606607429031361_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6071606607429031361_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6071606607429031361_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5952502267395132300_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5952502267395132300_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5952502267395132300_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8721375323659339135_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8721375323659339135_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8721375323659339135_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7950482702723893487_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7950482702723893487_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7950482702723893487_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7166449627492350013_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7166449627492350013_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7166449627492350013_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5017676924772438525_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:54 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5017676924772438525_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5017676924772438525_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8749207748679428914_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8749207748679428914_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8749207748679428914_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7520792284271291773_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7520792284271291773_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7520792284271291773_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7645583456694788719_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7645583456694788719_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7645583456694788719_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8102663884604135066_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8102663884604135066_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8102663884604135066_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8751462557539272242_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8751462557539272242_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8751462557539272242_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9155382214625873932_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9155382214625873932_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9155382214625873932_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7580724440894386303_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7580724440894386303_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7580724440894386303_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4853750465196267207_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4853750465196267207_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4853750465196267207_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8654771065013382890_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8654771065013382890_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8654771065013382890_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5933018796480200252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:55 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5933018796480200252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5933018796480200252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8748615651485508731_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8748615651485508731_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8748615651485508731_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6780038765493169806_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6780038765493169806_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6780038765493169806_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6274865418622646896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6274865418622646896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6274865418622646896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5135505344593799025_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5135505344593799025_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5135505344593799025_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8672689193868631016_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8672689193868631016_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8672689193868631016_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8191354233715868841_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8191354233715868841_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8191354233715868841_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7009284996917559051_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7009284996917559051_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7009284996917559051_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6174064074032009921_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6174064074032009921_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6174064074032009921_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6677316070679374403_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6677316070679374403_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6677316070679374403_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7959410391188701111_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:56 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7959410391188701111_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7959410391188701111_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4649279761003430060_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4649279761003430060_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4649279761003430060_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7135956371278745424_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7135956371278745424_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7135956371278745424_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8741356237717245994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8741356237717245994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8741356237717245994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7942741021595218288_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7942741021595218288_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7942741021595218288_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8872604074043318690_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8872604074043318690_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8872604074043318690_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5892450418636220799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5892450418636220799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5892450418636220799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7053354978498323172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7053354978498323172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7053354978498323172_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5785050929630342775_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5785050929630342775_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5785050929630342775_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8450375289350603684_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8450375289350603684_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8450375289350603684_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7390273941854027809_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:57 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7390273941854027809_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7390273941854027809_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5023925065400261865_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5023925065400261865_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5023925065400261865_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8125218423659763304_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8125218423659763304_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8125218423659763304_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8776211032688069555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8776211032688069555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8776211032688069555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7743744887933676669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7743744887933676669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7743744887933676669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7729191562624533856_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7729191562624533856_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7729191562624533856_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7774031599214612408_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7774031599214612408_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7774031599214612408_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6467654305871754221_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6467654305871754221_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6467654305871754221_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7505992142409431010_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7505992142409431010_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7505992142409431010_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5427596653825871670_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5427596653825871670_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5427596653825871670_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6219063934649806196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:58 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6219063934649806196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6219063934649806196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7375942959226980896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7375942959226980896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7375942959226980896_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5820149171655120522_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5820149171655120522_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5820149171655120522_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9004716786588964074_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9004716786588964074_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9004716786588964074_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4646148896086950163_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4646148896086950163_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4646148896086950163_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4797995107977245261_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4797995107977245261_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4797995107977245261_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7597564869130376602_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7597564869130376602_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7597564869130376602_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6959423968797355128_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6959423968797355128_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6959423968797355128_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5911106129903124238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5911106129903124238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5911106129903124238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7811971155504793146_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7811971155504793146_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7811971155504793146_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9098946068384666366_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:52:59 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9098946068384666366_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9098946068384666366_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5393816876045413178_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5393816876045413178_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5393816876045413178_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7684409117536607439_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7684409117536607439_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7684409117536607439_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5050089626293495744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5050089626293495744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5050089626293495744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6804809026714922209_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6804809026714922209_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6804809026714922209_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6753325580983055250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6753325580983055250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6753325580983055250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7266165704540648507_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7266165704540648507_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7266165704540648507_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6097045955446647250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6097045955446647250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6097045955446647250_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5008317531132840122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5008317531132840122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5008317531132840122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8353342067934170418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8353342067934170418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8353342067934170418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8242932336375392263_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:00 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8242932336375392263_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8242932336375392263_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6434553444799378512_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6434553444799378512_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6434553444799378512_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7713478836773603913_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7713478836773603913_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7713478836773603913_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8640764803597391756_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8640764803597391756_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8640764803597391756_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7247106239690589761_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7247106239690589761_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7247106239690589761_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6895172036835716569_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6895172036835716569_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6895172036835716569_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7907944515372473204_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7907944515372473204_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7907944515372473204_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8093831471865932833_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8093831471865932833_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8093831471865932833_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6979519332023183584_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6979519332023183584_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6979519332023183584_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5544437906181939637_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5544437906181939637_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5544437906181939637_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6791392727552430210_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:01 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6791392727552430210_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6791392727552430210_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5257403965418092799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5257403965418092799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5257403965418092799_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6109186720621162936_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6109186720621162936_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6109186720621162936_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8942471704622096045_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8942471704622096045_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8942471704622096045_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5820677694729368380_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5820677694729368380_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5820677694729368380_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5306728848982231112_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5306728848982231112_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5306728848982231112_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5338875729185184744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5338875729185184744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5338875729185184744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7603900825596662770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7603900825596662770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7603900825596662770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5042644793461417471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5042644793461417471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5042644793461417471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8342840504814040908_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8342840504814040908_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8342840504814040908_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8435935928085172805_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:02 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8435935928085172805_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8435935928085172805_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4975870410766644489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4975870410766644489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4975870410766644489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7029230750906510313_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7029230750906510313_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7029230750906510313_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6978735013438370639_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6978735013438370639_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6978735013438370639_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5890410016480964827_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5890410016480964827_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5890410016480964827_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5030247355532800947_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5030247355532800947_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5030247355532800947_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8037676132918071397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8037676132918071397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8037676132918071397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6355326253247637495_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6355326253247637495_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6355326253247637495_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6978836479730212432_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6978836479730212432_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6978836479730212432_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7061932707984923754_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7061932707984923754_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7061932707984923754_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5858938301638464804_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:03 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5858938301638464804_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5858938301638464804_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4952096433226868205_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4952096433226868205_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4952096433226868205_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5348210475835870998_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5348210475835870998_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5348210475835870998_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5473277745942104557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5473277745942104557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5473277745942104557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5687901463216273557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5687901463216273557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5687901463216273557_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7171474004797303608_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7171474004797303608_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7171474004797303608_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6075500450834383530_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6075500450834383530_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6075500450834383530_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8022548174306253107_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8022548174306253107_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8022548174306253107_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4965072912358269448_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4965072912358269448_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4965072912358269448_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5500622978029316912_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5500622978029316912_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5500622978029316912_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6583022547860789916_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:04 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6583022547860789916_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6583022547860789916_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4679296952132046277_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4679296952132046277_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4679296952132046277_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5980851969104385616_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5980851969104385616_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5980851969104385616_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5552150569745505532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5552150569745505532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5552150569745505532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6497410621362409679_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6497410621362409679_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6497410621362409679_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6480302362560079744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6480302362560079744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6480302362560079744_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8445206028403458251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8445206028403458251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8445206028403458251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5346842401792707563_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5346842401792707563_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5346842401792707563_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6566415107412013732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6566415107412013732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6566415107412013732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9035037997982150554_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9035037997982150554_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9035037997982150554_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5823643792381407656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:05 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5823643792381407656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5823643792381407656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8201172461547314285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8201172461547314285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8201172461547314285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8728497015209428460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8728497015209428460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8728497015209428460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6260761023741804751_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6260761023741804751_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6260761023741804751_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8759681441778720819_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8759681441778720819_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8759681441778720819_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7655871191889506974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7655871191889506974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7655871191889506974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4642021329039815252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4642021329039815252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4642021329039815252_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4683003894855570624_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4683003894855570624_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4683003894855570624_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5313036863556443452_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5313036863556443452_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5313036863556443452_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4701265451999725306_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4701265451999725306_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4701265451999725306_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8472811620330917007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:06 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8472811620330917007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8472811620330917007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7271809398884323958_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7271809398884323958_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7271809398884323958_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8461534096038668878_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8461534096038668878_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8461534096038668878_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6868513244393836040_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6868513244393836040_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6868513244393836040_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5561443643403711471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5561443643403711471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5561443643403711471_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5101106800401216689_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5101106800401216689_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5101106800401216689_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9059467284275720580_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9059467284275720580_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9059467284275720580_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7255838955743726873_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7255838955743726873_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7255838955743726873_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6313411600252925196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6313411600252925196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6313411600252925196_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8601390371793455536_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8601390371793455536_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8601390371793455536_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8887614720283064790_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:07 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8887614720283064790_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8887614720283064790_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6004354791468833868_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6004354791468833868_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6004354791468833868_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8494267427805648032_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8494267427805648032_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8494267427805648032_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7358422580889304418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7358422580889304418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7358422580889304418_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9114330005426440397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9114330005426440397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9114330005426440397_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7361229628672187238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7361229628672187238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7361229628672187238_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5401955598374179994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5401955598374179994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5401955598374179994_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5006893666139939664_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5006893666139939664_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5006893666139939664_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8068242883224441985_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8068242883224441985_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8068242883224441985_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5947888916374148285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5947888916374148285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5947888916374148285_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4825621992463479700_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:08 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4825621992463479700_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4825621992463479700_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7103789153875824832_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7103789153875824832_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7103789153875824832_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8210948628783381723_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8210948628783381723_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8210948628783381723_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5374537651855149707_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5374537651855149707_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5374537651855149707_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6222760668050078614_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6222760668050078614_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6222760668050078614_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5655965604419359353_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5655965604419359353_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5655965604419359353_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8008942085292351450_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8008942085292351450_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8008942085292351450_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7631367763918526120_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7631367763918526120_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7631367763918526120_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7304601471394424555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7304601471394424555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7304601471394424555_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8024149475446514929_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8024149475446514929_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8024149475446514929_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8801802659996548699_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:09 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8801802659996548699_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8801802659996548699_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7605158241426183206_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7605158241426183206_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7605158241426183206_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6591138179150214334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_6591138179150214334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6591138179150214334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8753375423811885451_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8753375423811885451_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8753375423811885451_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7236819302733412586_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7236819302733412586_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7236819302733412586_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9082540323639904133_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_9082540323639904133_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9082540323639904133_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5848617557740124078_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5848617557740124078_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5848617557740124078_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5628814558982409340_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5628814558982409340_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5628814558982409340_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7401190057570637122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7401190057570637122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7401190057570637122_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8965986069220987501_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8965986069220987501_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8965986069220987501_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6314055430777233184_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_6314055430777233184_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6314055430777233184_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7574907957658879256_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:10 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7574907957658879256_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7574907957658879256_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7806616804798013014_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7806616804798013014_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7806616804798013014_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7684552934064910954_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7684552934064910954_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7684552934064910954_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5454044708262696710_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5454044708262696710_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5454044708262696710_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8161317507418626907_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8161317507418626907_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8161317507418626907_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5890201344315321915_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5890201344315321915_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5890201344315321915_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4847331135184827043_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_4847331135184827043_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4847331135184827043_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7460523275305133083_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7460523275305133083_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7460523275305133083_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4656999278718127581_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_4656999278718127581_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4656999278718127581_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7954235274529245859_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7954235274529245859_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7954235274529245859_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8505169190608997241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:11 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8505169190608997241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8505169190608997241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8760458370662759429_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8760458370662759429_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8760458370662759429_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5808896826927869115_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5808896826927869115_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5808896826927869115_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8770800908457415091_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8770800908457415091_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8770800908457415091_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8414640236899592736_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8414640236899592736_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8414640236899592736_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5629234972935935542_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5629234972935935542_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5629234972935935542_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7797206792215273770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7797206792215273770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7797206792215273770_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6059250966061625928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6059250966061625928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6059250966061625928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6916334911563085476_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6916334911563085476_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6916334911563085476_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4748067356547445412_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4748067356547445412_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4748067356547445412_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5574610761307428657_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:12 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5574610761307428657_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5574610761307428657_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6305941715737104687_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6305941715737104687_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6305941715737104687_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8892220999529644460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8892220999529644460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8892220999529644460_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6785699034603793788_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6785699034603793788_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6785699034603793788_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5525511379654324251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5525511379654324251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5525511379654324251_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7762789072175278001_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7762789072175278001_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7762789072175278001_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5814217456593720933_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5814217456593720933_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5814217456593720933_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6215179605608469663_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6215179605608469663_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6215179605608469663_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4655668996135367757_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4655668996135367757_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4655668996135367757_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4962371162344990241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_4962371162344990241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4962371162344990241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7349827700812125390_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:13 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7349827700812125390_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7349827700812125390_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9032011621882920524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9032011621882920524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9032011621882920524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7795384712013498047_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7795384712013498047_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7795384712013498047_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8226077224685229562_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8226077224685229562_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8226077224685229562_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7877402761778786559_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7877402761778786559_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7877402761778786559_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7751906982203052398_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7751906982203052398_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7751906982203052398_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6299227440541844508_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6299227440541844508_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6299227440541844508_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7016426625904810464_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7016426625904810464_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7016426625904810464_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9079392364611273762_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9079392364611273762_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9079392364611273762_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7421820331603046900_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7421820331603046900_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7421820331603046900_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:14 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5008938819954758662_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5008938819954758662_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5008938819954758662_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8490712982182093259_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8490712982182093259_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8490712982182093259_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8896085723821365286_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8896085723821365286_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8896085723821365286_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9157040790051219241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9157040790051219241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9157040790051219241_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7737780551929429265_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7737780551929429265_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7737780551929429265_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7361222368650717532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7361222368650717532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7361222368650717532_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7468482902728960978_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7468482902728960978_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7468482902728960978_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8163187734195214057_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8163187734195214057_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8163187734195214057_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8786844691346011334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8786844691346011334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8786844691346011334_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5878025978762129235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:15 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5878025978762129235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5878025978762129235_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8540160126711022694_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8540160126711022694_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8540160126711022694_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6056378783790998946_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6056378783790998946_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6056378783790998946_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6457921226285477237_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6457921226285477237_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6457921226285477237_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5302403258489975640_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5302403258489975640_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5302403258489975640_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6417452006881148636_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6417452006881148636_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6417452006881148636_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6426926369524420006_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6426926369524420006_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6426926369524420006_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8180319586546668850_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8180319586546668850_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8180319586546668850_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7120350461628180698_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7120350461628180698_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7120350461628180698_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8084470032469497672_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8084470032469497672_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8084470032469497672_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8164748043250947413_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:16 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8164748043250947413_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8164748043250947413_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7029018676570869171_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7029018676570869171_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7029018676570869171_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8941915100258445882_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8941915100258445882_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8941915100258445882_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8586532709633149391_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8586532709633149391_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8586532709633149391_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6610070692337005709_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6610070692337005709_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6610070692337005709_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9052134379817740007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9052134379817740007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9052134379817740007_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9059888901654753656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_9059888901654753656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9059888901654753656_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7120746519664609941_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7120746519664609941_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7120746519664609941_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5227005488618859849_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5227005488618859849_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5227005488618859849_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6611938410425849732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_6611938410425849732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6611938410425849732_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5526676374244356524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:17 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5526676374244356524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5526676374244356524_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8925619770756813860_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8925619770756813860_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8925619770756813860_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5176245196401597526_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_5176245196401597526_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5176245196401597526_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8437704450849671187_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8437704450849671187_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8437704450849671187_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7401805178332779387_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7401805178332779387_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7401805178332779387_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7082159227935713254_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7082159227935713254_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7082159227935713254_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7752490905198914753_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7752490905198914753_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7752490905198914753_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8113425234811773458_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_8113425234811773458_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8113425234811773458_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7263334731102017489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 2386554180684781620_7263334731102017489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7263334731102017489_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_4773244748751564671_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_4773244748751564671_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_4773244748751564671_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8744592190123971928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:18 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8744592190123971928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8744592190123971928_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7191278723699884293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7191278723699884293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7191278723699884293_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7700994208662825974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7700994208662825974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7700994208662825974_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6381078044502395627_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:19 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_6381078044502395627_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6381078044502395627_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6822457478001222441_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_6822457478001222441_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6822457478001222441_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8959603295698191591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8959603295698191591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:20 INFO HiveMetaStore: 13: get_database: trusted 23/06/28 07:53:20 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:53:20 INFO HiveMetaStore: 13: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:53:20 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:53:20 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:53:20 INFO HiveMetaStore: 13: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:53:20 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:53:21 INFO ClusterLoadMonitor: Added query with execution ID:331. Current active queries:1 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:21.185Z","run":{"runId":"768468e9-8235-408a-a0d3-5da580b074ea","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.GlobalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.LocalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:21 INFO SparkContext: Starting job: jdbc at command-1577463665195040:15 23/06/28 07:53:21 INFO DAGScheduler: Job 89 finished: jdbc at command-1577463665195040:15, took 0.000213 s 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO ClusterLoadMonitor: Removed query with execution ID:331. Current active queries:0 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:21 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:21 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:21 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:21 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:21 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8959603295698191591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_9143859025418618714_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_9143859025418618714_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_9143859025418618714_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8989233836769921669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8989233836769921669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:21 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:21.505Z","run":{"runId":"768468e9-8235-408a-a0d3-5da580b074ea","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.GlobalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.LocalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:21 INFO SQLExecution: Received onJobStart for query with unregistered executionId 331 23/06/28 07:53:22 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:22 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:22 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:21.505Z","run":{"runId":"768468e9-8235-408a-a0d3-5da580b074ea","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.GlobalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.LocalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:22 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:22 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:22 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:21.557Z","run":{"runId":"768468e9-8235-408a-a0d3-5da580b074ea","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.GlobalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.LocalLimit","num-children":1,"limitExpr":[{"class":"org.apache.spark.sql.catalyst.expressions.Literal","num-children":0,"value":"0","dataType":"integer"}],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6400,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6401,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6402,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6403,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6404,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6405,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6406,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6407,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6408,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6409,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6410,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6411,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:22 INFO ClusterLoadMonitor: Added query with execution ID:332. Current active queries:1 23/06/28 07:53:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:53:22 INFO CodeGenerator: Code generated in 17.526151 ms 23/06/28 07:53:22 INFO DAGScheduler: Registering RDD 448 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) as input to shuffle 50 23/06/28 07:53:22 INFO DAGScheduler: Got map stage job 90 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) with 1 output partitions 23/06/28 07:53:22 INFO DAGScheduler: Final stage: ShuffleMapStage 205 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) 23/06/28 07:53:22 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:22 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:22 INFO DAGScheduler: Submitting ShuffleMapStage 205 (MapPartitionsRDD[448] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604), which has no missing parents 23/06/28 07:53:22 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:53:22 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:22 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:22 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:22 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 205 (MapPartitionsRDD[448] at $anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:53:22 INFO TaskSchedulerImpl: Adding task set 205.0 with 1 tasks resource profile 0 23/06/28 07:53:22 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 4015519563002992008, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 4015519563002992008. Created 4015519563002992008 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:53:22 INFO FairSchedulableBuilder: Added task set TaskSet_205.0 tasks to pool 4015519563002992008 23/06/28 07:53:22 INFO TaskSetManager: Starting task 0.0 in stage 205.0 (TID 1575) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:22 INFO MemoryStore: Block broadcast_135 stored as values in memory (estimated size 133.4 KiB, free 750.6 GiB) 23/06/28 07:53:22 INFO MemoryStore: Block broadcast_135_piece0 stored as bytes in memory (estimated size 23.5 KiB, free 750.6 GiB) 23/06/28 07:53:22 INFO BlockManagerInfo: Added broadcast_135_piece0 in memory on 10.60.24.15:40507 (size: 23.5 KiB, free: 750.7 GiB) 23/06/28 07:53:22 INFO SparkContext: Created broadcast 135 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:22 INFO BlockManagerInfo: Added broadcast_135_piece0 in memory on 10.60.24.13:44035 (size: 23.5 KiB, free: 487.7 GiB) 23/06/28 07:53:22 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.0, New Ema: 1.0 23/06/28 07:53:25 INFO TaskSetManager: Finished task 0.0 in stage 205.0 (TID 1575) in 3162 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:53:25 INFO TaskSchedulerImpl: Removed TaskSet 205.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:25 INFO DAGScheduler: ShuffleMapStage 205 ($anonfun$withThreadLocalCaptured$1 at CompletableFuture.java:1604) finished in 3.168 s 23/06/28 07:53:25 INFO DAGScheduler: looking for newly runnable stages 23/06/28 07:53:25 INFO DAGScheduler: running: Set() 23/06/28 07:53:25 INFO DAGScheduler: waiting: Set() 23/06/28 07:53:25 INFO DAGScheduler: failed: Set() 23/06/28 07:53:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobEnd 23/06/28 07:53:25 INFO CodeGenerator: Code generated in 16.302906 ms 23/06/28 07:53:25 INFO SparkContext: Starting job: count at command-1577463665195041:11 23/06/28 07:53:25 INFO DAGScheduler: Got job 91 (count at command-1577463665195041:11) with 1 output partitions 23/06/28 07:53:25 INFO DAGScheduler: Final stage: ResultStage 207 (count at command-1577463665195041:11) 23/06/28 07:53:25 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 206) 23/06/28 07:53:25 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:25 INFO DAGScheduler: Submitting ResultStage 207 (MapPartitionsRDD[450] at count at command-1577463665195041:11), which has no missing parents 23/06/28 07:53:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerJobStart 23/06/28 07:53:25 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:25 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:25 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:25 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 207 (MapPartitionsRDD[450] at count at command-1577463665195041:11) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:53:25 INFO TaskSchedulerImpl: Adding task set 207.0 with 1 tasks resource profile 0 23/06/28 07:53:25 INFO FairSchedulableBuilder: Added task set TaskSet_207.0 tasks to pool 4015519563002992008 23/06/28 07:53:25 INFO TaskSetManager: Starting task 0.0 in stage 207.0 (TID 1576) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:25 INFO MemoryStore: Block broadcast_136 stored as values in memory (estimated size 129.8 KiB, free 750.6 GiB) 23/06/28 07:53:25 INFO MemoryStore: Block broadcast_136_piece0 stored as bytes in memory (estimated size 21.4 KiB, free 750.6 GiB) 23/06/28 07:53:25 INFO BlockManagerInfo: Added broadcast_136_piece0 in memory on 10.60.24.15:40507 (size: 21.4 KiB, free: 750.7 GiB) 23/06/28 07:53:25 INFO SparkContext: Created broadcast 136 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:25 INFO BlockManagerInfo: Added broadcast_136_piece0 in memory on 10.60.24.10:41041 (size: 21.4 KiB, free: 487.7 GiB) 23/06/28 07:53:25 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 50 to 10.60.24.10:44288 23/06/28 07:53:25 INFO TaskSetManager: Finished task 0.0 in stage 207.0 (TID 1576) in 24 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:53:25 INFO TaskSchedulerImpl: Removed TaskSet 207.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:25 INFO DAGScheduler: ResultStage 207 (count at command-1577463665195041:11) finished in 0.026 s 23/06/28 07:53:25 INFO DAGScheduler: Job 91 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:25 INFO TaskSchedulerImpl: Killing all running tasks in stage 207: Stage finished 23/06/28 07:53:25 INFO DAGScheduler: Job 91 finished: count at command-1577463665195041:11, took 0.027941 s 23/06/28 07:53:25 INFO ClusterLoadMonitor: Removed query with execution ID:332. Current active queries:0 23/06/28 07:53:25 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:53:25 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8989233836769921669_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:25 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8095818774189832603_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:25 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8095818774189832603_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:25 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:53:26 INFO ClusterLoadMonitor: Added query with execution ID:333. Current active queries:1 23/06/28 07:53:26 INFO CodeGenerator: Code generated in 10.404407 ms 23/06/28 07:53:26 INFO SparkContext: Starting job: collect at command-1577463665195038:26 23/06/28 07:53:26 INFO DAGScheduler: Got job 92 (collect at command-1577463665195038:26) with 1 output partitions 23/06/28 07:53:26 INFO DAGScheduler: Final stage: ResultStage 208 (collect at command-1577463665195038:26) 23/06/28 07:53:26 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:26 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:26 INFO DAGScheduler: Submitting ResultStage 208 (MapPartitionsRDD[452] at collect at command-1577463665195038:26), which has no missing parents 23/06/28 07:53:26 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:26 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:26 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:26 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 208 (MapPartitionsRDD[452] at collect at command-1577463665195038:26) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:53:26 INFO TaskSchedulerImpl: Adding task set 208.0 with 1 tasks resource profile 0 23/06/28 07:53:26 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 4015519563002992008, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 4015519563002992008. Created 4015519563002992008 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:53:26 INFO FairSchedulableBuilder: Added task set TaskSet_208.0 tasks to pool 4015519563002992008 23/06/28 07:53:26 INFO TaskSetManager: Starting task 0.0 in stage 208.0 (TID 1577) (10.60.24.13, executor 1, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:26 INFO MemoryStore: Block broadcast_137 stored as values in memory (estimated size 129.6 KiB, free 750.6 GiB) 23/06/28 07:53:26 INFO MemoryStore: Block broadcast_137_piece0 stored as bytes in memory (estimated size 21.7 KiB, free 750.6 GiB) 23/06/28 07:53:26 INFO BlockManagerInfo: Added broadcast_137_piece0 in memory on 10.60.24.15:40507 (size: 21.7 KiB, free: 750.7 GiB) 23/06/28 07:53:26 INFO SparkContext: Created broadcast 137 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:26 INFO BlockManagerInfo: Added broadcast_137_piece0 in memory on 10.60.24.13:44035 (size: 21.7 KiB, free: 487.7 GiB) 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:26.094Z","run":{"runId":"6c1804eb-c82d-4b76-b8fb-516cbacb3bd2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"column_name","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6540,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_scan_jdbc_relation((_selec_t_upper(column_name)_column_name_fro_m_informatio_n_schema__column_s_wher_e_tabl_e_schem_a = 'stage'_an_d_tabl_e_nam_e = '_lm_s_lmsuu_mdl_groups_members')_qry) [num_partitions=1] ","facets":{}},"inputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"INFORMATION_SCHEMA.COLUMNS","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"column_name","type":"string"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:26 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:26 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:26 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:26 INFO TaskSetManager: Finished task 0.0 in stage 208.0 (TID 1577) in 98 ms on 10.60.24.13 (executor 1) (1/1) 23/06/28 07:53:26 INFO TaskSchedulerImpl: Removed TaskSet 208.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:26 INFO DAGScheduler: ResultStage 208 (collect at command-1577463665195038:26) finished in 0.100 s 23/06/28 07:53:26 INFO DAGScheduler: Job 92 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:26 INFO TaskSchedulerImpl: Killing all running tasks in stage 208: Stage finished 23/06/28 07:53:26 INFO DAGScheduler: Job 92 finished: collect at command-1577463665195038:26, took 0.101610 s 23/06/28 07:53:26 INFO CodeGenerator: Code generated in 8.882217 ms 23/06/28 07:53:26 INFO ClusterLoadMonitor: Removed query with execution ID:333. Current active queries:0 23/06/28 07:53:26 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:26.108Z","run":{"runId":"6c1804eb-c82d-4b76-b8fb-516cbacb3bd2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"column_name","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6540,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_scan_jdbc_relation((_selec_t_upper(column_name)_column_name_fro_m_informatio_n_schema__column_s_wher_e_tabl_e_schem_a = 'stage'_an_d_tabl_e_nam_e = '_lm_s_lmsuu_mdl_groups_members')_qry) [num_partitions=1] ","facets":{}},"inputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"INFORMATION_SCHEMA.COLUMNS","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"column_name","type":"string"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:26 INFO SQLExecution: Received onJobStart for query with unregistered executionId 333 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:26.209Z","run":{"runId":"6c1804eb-c82d-4b76-b8fb-516cbacb3bd2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"column_name","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6540,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_scan_jdbc_relation((_selec_t_upper(column_name)_column_name_fro_m_informatio_n_schema__column_s_wher_e_tabl_e_schem_a = 'stage'_an_d_tabl_e_nam_e = '_lm_s_lmsuu_mdl_groups_members')_qry) [num_partitions=1] ","facets":{}},"inputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"INFORMATION_SCHEMA.COLUMNS","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"column_name","type":"string"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:26 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:26 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:26.232Z","run":{"runId":"6c1804eb-c82d-4b76-b8fb-516cbacb3bd2","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"column_name","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6540,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_scan_jdbc_relation((_selec_t_upper(column_name)_column_name_fro_m_informatio_n_schema__column_s_wher_e_tabl_e_schem_a = 'stage'_an_d_tabl_e_nam_e = '_lm_s_lmsuu_mdl_groups_members')_qry) [num_partitions=1] ","facets":{}},"inputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"INFORMATION_SCHEMA.COLUMNS","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"column_name","type":"string"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8095818774189832603_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_8736483795246241591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_8736483795246241591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_8736483795246241591_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_6332321228941599015_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_6332321228941599015_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_6332321228941599015_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_7923147120299192743_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_7923147120299192743_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:27 INFO HiveMetaStore: 14: get_database: trusted 23/06/28 07:53:27 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: trusted 23/06/28 07:53:27 INFO HiveMetaStore: 14: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:53:27 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:53:27 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:53:27 INFO HiveMetaStore: 14: get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:53:27 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=trusted tbls=lms_lmsuu_mdl_groups_members 23/06/28 07:53:27 INFO ClusterLoadMonitor: Added query with execution ID:334. Current active queries:1 23/06/28 07:53:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:27 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:27.952Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:28 INFO SnapshotEdge: [tableId=5d0ae5b6-42a3-47d1-8d94-be2429376ace] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_138 stored as values in memory (estimated size 508.5 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_138_piece0 stored as bytes in memory (estimated size 13.6 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_138_piece0 in memory on 10.60.24.15:40507 (size: 13.6 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 138 from broadcast at DeltaLog.scala:634 23/06/28 07:53:28 INFO CodeGenerator: Code generated in 36.246057 ms 23/06/28 07:53:28 INFO CodeGenerator: Code generated in 23.527559 ms 23/06/28 07:53:28 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:53:28 INFO DAGScheduler: Got job 93 (executeCollect at DatasetRefCache.scala:71) with 2 output partitions 23/06/28 07:53:28 INFO DAGScheduler: Final stage: ResultStage 209 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:53:28 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:28 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:28 INFO DAGScheduler: Submitting ResultStage 209 (MapPartitionsRDD[455] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:28 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:28 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 209 (MapPartitionsRDD[455] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0, 1)) 23/06/28 07:53:28 INFO TaskSchedulerImpl: Adding task set 209.0 with 2 tasks resource profile 0 23/06/28 07:53:28 INFO FairSchedulableBuilder: Added task set TaskSet_209.0 tasks to pool 4015519563002992008 23/06/28 07:53:28 INFO TaskSetManager: Starting task 0.0 in stage 209.0 (TID 1578) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 1.0 in stage 209.0 (TID 1579) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_139 stored as values in memory (estimated size 209.5 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_139_piece0 stored as bytes in memory (estimated size 44.8 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_139_piece0 in memory on 10.60.24.15:40507 (size: 44.8 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 139 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_139_piece0 in memory on 10.60.24.10:41041 (size: 44.8 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_139_piece0 in memory on 10.60.24.13:44035 (size: 44.8 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 0.0 in stage 209.0 (TID 1578) in 21 ms on 10.60.24.10 (executor 0) (1/2) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 1.0 in stage 209.0 (TID 1579) in 42 ms on 10.60.24.13 (executor 1) (2/2) 23/06/28 07:53:28 INFO TaskSchedulerImpl: Removed TaskSet 209.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:28 INFO DAGScheduler: ResultStage 209 (executeCollect at DatasetRefCache.scala:71) finished in 0.049 s 23/06/28 07:53:28 INFO DAGScheduler: Job 93 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:28 INFO TaskSchedulerImpl: Killing all running tasks in stage 209: Stage finished 23/06/28 07:53:28 INFO DAGScheduler: Job 93 finished: executeCollect at DatasetRefCache.scala:71, took 0.050352 s 23/06/28 07:53:28 INFO PropagateEmptyRelation: Removed leaf nodes: - LocalRelation , [path#6711, partitionValues#6712, size#6713L, modificationTime#6714L, dataChange#6715, tags#6716, deletionVector#6717, stats_parsed#6718, version#6719L] 23/06/28 07:53:28 INFO FileSourceStrategy: Pushed Filters: IsNotNull(add.dataChange) 23/06/28 07:53:28 INFO FileSourceStrategy: Post-Scan Filters: isnotnull(add#6632.dataChange) 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO CodeGenerator: Code generated in 34.14548 ms 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_140 stored as values in memory (estimated size 598.5 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_140_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_140_piece0 in memory on 10.60.24.15:40507 (size: 16.4 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 140 from executeCollect at DatasetRefCache.scala:71 23/06/28 07:53:28 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:53:28 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:53:28 INFO DAGScheduler: Got job 94 (executeCollect at DatasetRefCache.scala:71) with 1 output partitions 23/06/28 07:53:28 INFO DAGScheduler: Final stage: ResultStage 210 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:53:28 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:28 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:28 INFO DAGScheduler: Submitting ResultStage 210 (MapPartitionsRDD[458] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:53:28 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:28 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:28 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:28 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 210 (MapPartitionsRDD[458] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:53:28 INFO TaskSchedulerImpl: Adding task set 210.0 with 1 tasks resource profile 0 23/06/28 07:53:28 INFO FairSchedulableBuilder: Added task set TaskSet_210.0 tasks to pool 4015519563002992008 23/06/28 07:53:28 INFO TaskSetManager: Starting task 0.0 in stage 210.0 (TID 1580) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_141 stored as values in memory (estimated size 260.9 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_141_piece0 stored as bytes in memory (estimated size 60.2 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_141_piece0 in memory on 10.60.24.15:40507 (size: 60.2 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 141 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:28 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:28.17Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_141_piece0 in memory on 10.60.24.10:41041 (size: 60.2 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_140_piece0 in memory on 10.60.24.10:41041 (size: 16.4 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:28.219Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{"outputStatistics":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/OutputStatisticsOutputDatasetFacet.json#/$defs/OutputStatisticsOutputDatasetFacet","rowCount":0,"size":0}}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:28.295Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:28 INFO TaskSetManager: Finished task 0.0 in stage 210.0 (TID 1580) in 230 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:53:28 INFO TaskSchedulerImpl: Removed TaskSet 210.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:28 INFO DAGScheduler: ResultStage 210 (executeCollect at DatasetRefCache.scala:71) finished in 0.233 s 23/06/28 07:53:28 INFO DAGScheduler: Job 94 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:28 INFO TaskSchedulerImpl: Killing all running tasks in stage 210: Stage finished 23/06/28 07:53:28 INFO DAGScheduler: Job 94 finished: executeCollect at DatasetRefCache.scala:71, took 0.234621 s 23/06/28 07:53:28 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionStart 23/06/28 07:53:28 INFO CodeGenerator: Code generated in 14.423508 ms 23/06/28 07:53:28 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:53:28 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:53:28 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:53:28 INFO CodeGenerator: Code generated in 8.909024 ms 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_142 stored as values in memory (estimated size 557.8 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_142_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_142_piece0 in memory on 10.60.24.15:40507 (size: 15.3 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 142 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:53:28 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:53:28 INFO SparkContext: Starting job: jdbc at command-1577463665195040:15 23/06/28 07:53:28 INFO DAGScheduler: Got job 95 (jdbc at command-1577463665195040:15) with 6 output partitions 23/06/28 07:53:28 INFO DAGScheduler: Final stage: ResultStage 211 (jdbc at command-1577463665195040:15) 23/06/28 07:53:28 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:28 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:28 INFO DAGScheduler: Submitting ResultStage 211 (MapPartitionsRDD[464] at jdbc at command-1577463665195040:15), which has no missing parents 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:28 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:28 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:28 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 211 (MapPartitionsRDD[464] at jdbc at command-1577463665195040:15) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5)) 23/06/28 07:53:28 INFO TaskSchedulerImpl: Adding task set 211.0 with 6 tasks resource profile 0 23/06/28 07:53:28 INFO FairSchedulableBuilder: Added task set TaskSet_211.0 tasks to pool 4015519563002992008 23/06/28 07:53:28 INFO TaskSetManager: Starting task 4.0 in stage 211.0 (TID 1581) (10.60.24.10, executor 0, partition 4, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 0.0 in stage 211.0 (TID 1582) (10.60.24.13, executor 1, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 5.0 in stage 211.0 (TID 1583) (10.60.24.10, executor 0, partition 5, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 1.0 in stage 211.0 (TID 1584) (10.60.24.13, executor 1, partition 1, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 2.0 in stage 211.0 (TID 1585) (10.60.24.13, executor 1, partition 2, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO TaskSetManager: Starting task 3.0 in stage 211.0 (TID 1586) (10.60.24.13, executor 1, partition 3, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_143 stored as values in memory (estimated size 209.9 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO MemoryStore: Block broadcast_143_piece0 stored as bytes in memory (estimated size 57.5 KiB, free 750.6 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_143_piece0 in memory on 10.60.24.15:40507 (size: 57.5 KiB, free: 750.7 GiB) 23/06/28 07:53:28 INFO SparkContext: Created broadcast 143 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_143_piece0 in memory on 10.60.24.13:44035 (size: 57.5 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_143_piece0 in memory on 10.60.24.10:41041 (size: 57.5 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:28 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:28 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:28 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_142_piece0 in memory on 10.60.24.13:44035 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 3.0 in stage 211.0 (TID 1586) in 95 ms on 10.60.24.13 (executor 1) (1/6) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 0.0 in stage 211.0 (TID 1582) in 95 ms on 10.60.24.13 (executor 1) (2/6) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 1.0 in stage 211.0 (TID 1584) in 96 ms on 10.60.24.13 (executor 1) (3/6) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 2.0 in stage 211.0 (TID 1585) in 96 ms on 10.60.24.13 (executor 1) (4/6) 23/06/28 07:53:28 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:28.61Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:28 INFO BlockManagerInfo: Added broadcast_142_piece0 in memory on 10.60.24.10:41041 (size: 15.3 KiB, free: 487.7 GiB) 23/06/28 07:53:28 INFO TaskSetManager: Finished task 5.0 in stage 211.0 (TID 1583) in 220 ms on 10.60.24.10 (executor 0) (5/6) 23/06/28 07:53:28 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 0.85, New Ema: 1.0 23/06/28 07:53:30 INFO TaskSetManager: Finished task 4.0 in stage 211.0 (TID 1581) in 1489 ms on 10.60.24.10 (executor 0) (6/6) 23/06/28 07:53:30 INFO TaskSchedulerImpl: Removed TaskSet 211.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:30 INFO DAGScheduler: ResultStage 211 (jdbc at command-1577463665195040:15) finished in 1.496 s 23/06/28 07:53:30 INFO DAGScheduler: Job 95 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:30 INFO TaskSchedulerImpl: Killing all running tasks in stage 211: Stage finished 23/06/28 07:53:30 INFO DAGScheduler: Job 95 finished: jdbc at command-1577463665195040:15, took 1.497787 s 23/06/28 07:53:30 INFO ClusterLoadMonitor: Removed query with execution ID:334. Current active queries:0 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:30.158Z","run":{"runId":"2ac4afcf-3113-4013-9832-18c2e1848794","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand","num-children":0,"query":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, trusted, LMS_lmsuu_mdl_groups_members]"}]],"child":0},{"class":"org.apache.spark.sql.catalyst.plans.logical.SubqueryAlias","num-children":1,"identifier":{"product-class":"org.apache.spark.sql.catalyst.AliasIdentifier","name":"LMS_lmsuu_mdl_groups_members","qualifier":"[spark_catalog, trusted]"},"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"component","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6557,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"groupid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6558,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6559,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"itemid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6560,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"timeadded","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6561,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"userid","dataType":"long","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6562,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLRawZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6563,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_DLTrustedZoneTimeStamp","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6564,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordStart","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6565,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordEnd","dataType":"timestamp","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6566,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordDeleted","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6567,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"_RecordCurrent","dataType":"integer","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6568,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members","database":"trusted","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"EXTERNAL"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"component","type":"string","nullable":true,"metadata":{}},{"name":"groupid","type":"long","nullable":true,"metadata":{}},{"name":"id","type":"long","nullable":true,"metadata":{}},{"name":"itemid","type":"long","nullable":true,"metadata":{}},{"name":"timeadded","type":"long","nullable":true,"metadata":{}},{"name":"userid","type":"long","nullable":true,"metadata":{}},{"name":"_DLRawZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordStart","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordEnd","type":"timestamp","nullable":true,"metadata":{}},{"name":"_RecordDeleted","type":"integer","nullable":true,"metadata":{}},{"name":"_RecordCurrent","type":"integer","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1642649720000,"lastAccessTime":0,"createVersion":"3.1.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}],"dataSource":null,"options":null,"mode":null}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_save_into_data_source_command_stage_LMS_lmsuu_mdl_groups_members","facets":{}},"inputs":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members","name":"spark_catalog.trusted.lms_lmsuu_mdl_groups_members","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[{"namespace":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","name":"stage.LMS_lmsuu_mdl_groups_members","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01","uri":"sqlserver://zesqlbi01t.database.windows.net:1433;database=ZEASQLDBEDW01"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"component","type":"string"},{"name":"groupid","type":"long"},{"name":"id","type":"long"},{"name":"itemid","type":"long"},{"name":"timeadded","type":"long"},{"name":"userid","type":"long"},{"name":"_DLRawZoneTimeStamp","type":"timestamp"},{"name":"_DLTrustedZoneTimeStamp","type":"timestamp"},{"name":"_RecordStart","type":"timestamp"},{"name":"_RecordEnd","type":"timestamp"},{"name":"_RecordDeleted","type":"integer"},{"name":"_RecordCurrent","type":"integer"}]},"columnLineage":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json#/$defs/ColumnLineageDatasetFacet","fields":{"component":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"component"}]},"groupid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"groupid"}]},"id":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"id"}]},"itemid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"itemid"}]},"timeadded":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"timeadded"}]},"userid":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"userid"}]},"_DLRawZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLRawZoneTimeStamp"}]},"_DLTrustedZoneTimeStamp":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_DLTrustedZoneTimeStamp"}]},"_RecordStart":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordStart"}]},"_RecordEnd":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordEnd"}]},"_RecordDeleted":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordDeleted"}]},"_RecordCurrent":{"inputFields":[{"namespace":"dbfs","name":"/mnt/datalake-trusted/lms/lms_lmsuu_mdl_groups_members/delta","field":"_RecordCurrent"}]}}},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"OVERWRITE"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:30 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_7923147120299192743_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:30 INFO ProgressReporter$: Removed result fetcher for 2386554180684781620_5380942922734008000_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:30 INFO ProgressReporter$: Added result fetcher for 4015519563002992008_5380942922734008000_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:30 INFO HiveMetaStore: 15: get_database: default 23/06/28 07:53:30 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:53:30 INFO HiveMetaStore: 15: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:53:30 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:53:30 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:53:30 INFO HiveMetaStore: 15: get_multi_table : db=default tbls=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:30 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=default tbls=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:30 INFO ClusterLoadMonitor: Added query with execution ID:336. Current active queries:1 23/06/28 07:53:30 INFO SnapshotEdge: [tableId=0a243321-de86-4bc0-9520-53c494e03423] Prefetch checkpoint is disabled; falling back to read checkpoint with Spark job 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_144 stored as values in memory (estimated size 508.5 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_144_piece0 stored as bytes in memory (estimated size 13.6 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_144_piece0 in memory on 10.60.24.15:40507 (size: 13.6 KiB, free: 750.7 GiB) 23/06/28 07:53:30 INFO SparkContext: Created broadcast 144 from broadcast at DeltaLog.scala:634 23/06/28 07:53:30 INFO SparkContext: Starting job: executeCollect at DatasetRefCache.scala:71 23/06/28 07:53:30 INFO DAGScheduler: Got job 96 (executeCollect at DatasetRefCache.scala:71) with 4 output partitions 23/06/28 07:53:30 INFO DAGScheduler: Final stage: ResultStage 212 (executeCollect at DatasetRefCache.scala:71) 23/06/28 07:53:30 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:30 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:30 INFO DAGScheduler: Submitting ResultStage 212 (MapPartitionsRDD[467] at executeCollect at DatasetRefCache.scala:71), which has no missing parents 23/06/28 07:53:30 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:30 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:30 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:30 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 212 (MapPartitionsRDD[467] at executeCollect at DatasetRefCache.scala:71) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 23/06/28 07:53:30 INFO TaskSchedulerImpl: Adding task set 212.0 with 4 tasks resource profile 0 23/06/28 07:53:30 WARN FairSchedulableBuilder: A job was submitted with scheduler pool 4015519563002992008, which has not been configured. This can happen when the file that pools are read from isn't set, or when that file doesn't contain 4015519563002992008. Created 4015519563002992008 with default configuration (schedulingMode: FIFO, minShare: 0, weight: 1) 23/06/28 07:53:30 INFO FairSchedulableBuilder: Added task set TaskSet_212.0 tasks to pool 4015519563002992008 23/06/28 07:53:30 INFO TaskSetManager: Starting task 0.0 in stage 212.0 (TID 1587) (10.60.24.10, executor 0, partition 0, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:30 INFO TaskSetManager: Starting task 1.0 in stage 212.0 (TID 1588) (10.60.24.13, executor 1, partition 1, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:30 INFO TaskSetManager: Starting task 2.0 in stage 212.0 (TID 1589) (10.60.24.10, executor 0, partition 2, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:30 INFO TaskSetManager: Starting task 3.0 in stage 212.0 (TID 1590) (10.60.24.13, executor 1, partition 3, PROCESS_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_145 stored as values in memory (estimated size 208.1 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_145_piece0 stored as bytes in memory (estimated size 44.3 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_145_piece0 in memory on 10.60.24.15:40507 (size: 44.3 KiB, free: 750.7 GiB) 23/06/28 07:53:30 INFO SparkContext: Created broadcast 145 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_145_piece0 in memory on 10.60.24.13:44035 (size: 44.3 KiB, free: 487.7 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_145_piece0 in memory on 10.60.24.10:41041 (size: 44.3 KiB, free: 487.7 GiB) 23/06/28 07:53:30 INFO TaskSetManager: Finished task 1.0 in stage 212.0 (TID 1588) in 35 ms on 10.60.24.13 (executor 1) (1/4) 23/06/28 07:53:30 INFO TaskSetManager: Finished task 3.0 in stage 212.0 (TID 1590) in 37 ms on 10.60.24.13 (executor 1) (2/4) 23/06/28 07:53:30 INFO TaskSetManager: Finished task 2.0 in stage 212.0 (TID 1589) in 40 ms on 10.60.24.10 (executor 0) (3/4) 23/06/28 07:53:30 INFO TaskSetManager: Finished task 0.0 in stage 212.0 (TID 1587) in 41 ms on 10.60.24.10 (executor 0) (4/4) 23/06/28 07:53:30 INFO TaskSchedulerImpl: Removed TaskSet 212.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:30 INFO DAGScheduler: ResultStage 212 (executeCollect at DatasetRefCache.scala:71) finished in 0.047 s 23/06/28 07:53:30 INFO DAGScheduler: Job 96 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:30 INFO TaskSchedulerImpl: Killing all running tasks in stage 212: Stage finished 23/06/28 07:53:30 INFO DAGScheduler: Job 96 finished: executeCollect at DatasetRefCache.scala:71, took 0.048240 s 23/06/28 07:53:30 INFO CodeGenerator: Code generated in 15.9783 ms 23/06/28 07:53:30 INFO FileSourceStrategy: Pushed Filters: 23/06/28 07:53:30 INFO FileSourceStrategy: Post-Scan Filters: 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:30 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:30 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:30 INFO CodeGenerator: Code generated in 7.893324 ms 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_146 stored as values in memory (estimated size 535.3 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_146_piece0 stored as bytes in memory (estimated size 14.9 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_146_piece0 in memory on 10.60.24.15:40507 (size: 14.9 KiB, free: 750.7 GiB) 23/06/28 07:53:30 INFO SparkContext: Created broadcast 146 from buildReaderWithPartitionValues at DeltaParquetFileFormat.scala:100 23/06/28 07:53:30 INFO FileSourceScanExec: Planning scan with bin packing, max split size: 134217728 bytes, max partition size: 4194304, open cost is considered as scanning 4194304 bytes. 23/06/28 07:53:30 INFO SparkContext: Starting job: collect at command-1577463665195035:9 23/06/28 07:53:30 INFO DAGScheduler: Got job 97 (collect at command-1577463665195035:9) with 1 output partitions 23/06/28 07:53:30 INFO DAGScheduler: Final stage: ResultStage 213 (collect at command-1577463665195035:9) 23/06/28 07:53:30 INFO DAGScheduler: Parents of final stage: List() 23/06/28 07:53:30 INFO DAGScheduler: Missing parents: List() 23/06/28 07:53:30 INFO DAGScheduler: Submitting ResultStage 213 (MapPartitionsRDD[470] at collect at command-1577463665195035:9), which has no missing parents 23/06/28 07:53:30 INFO DAGScheduler: Jars for session None: Map(spark://10.60.24.15:37693/jars/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372040, spark://10.60.24.15:37693/jars/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371866, spark://10.60.24.15:37693/jars/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371785, spark://10.60.24.15:37693/jars/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371878, spark://10.60.24.15:37693/jars/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938372006, spark://10.60.24.15:37693/jars/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371772, spark://10.60.24.15:37693/jars/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371935, spark://10.60.24.15:37693/jars/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372117, spark://10.60.24.15:37693/jars/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371912, spark://10.60.24.15:37693/jars/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371949, spark://10.60.24.15:37693/jars/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372074, spark://10.60.24.15:37693/jars/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372062, spark://10.60.24.15:37693/jars/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371809, spark://10.60.24.15:37693/jars/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371855, spark://10.60.24.15:37693/jars/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371758, spark://10.60.24.15:37693/jars/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371820, spark://10.60.24.15:37693/jars/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371889, spark://10.60.24.15:37693/jars/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372106, spark://10.60.24.15:37693/jars/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371746, spark://10.60.24.15:37693/jars/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371983, spark://10.60.24.15:37693/jars/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371900, spark://10.60.24.15:37693/jars/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371960, spark://10.60.24.15:37693/jars/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371797, spark://10.60.24.15:37693/jars/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372017, spark://10.60.24.15:37693/jars/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372096, spark://10.60.24.15:37693/jars/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371972, spark://10.60.24.15:37693/jars/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372028, spark://10.60.24.15:37693/jars/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372086, spark://10.60.24.15:37693/jars/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372051, spark://10.60.24.15:37693/jars/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371832, spark://10.60.24.15:37693/jars/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371923, spark://10.60.24.15:37693/jars/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371732, spark://10.60.24.15:37693/jars/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371843, spark://10.60.24.15:37693/jars/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371994) 23/06/28 07:53:30 INFO DAGScheduler: Files for session None: Map(spark://10.60.24.15:37693/files/addedFile8392563531864407349org_apache_maven_maven_model_2_2_1-01c48.jar -> 1687938372065, spark://10.60.24.15:37693/files/addedFile3479084762534129249org_sonatype_plexus_plexus_sec_dispatcher_1_4-c2c18.jar -> 1687938371869, spark://10.60.24.15:37693/files/addedFile3739801946456650712commons_codec_commons_codec_1_10-eb454.jar -> 1687938371986, spark://10.60.24.15:37693/files/addedFile4002780616708205610org_sonatype_plexus_plexus_cipher_1_4-91fb5.jar -> 1687938371892, spark://10.60.24.15:37693/files/addedFile6415962189126386174javax_activation_activation_1_1-29e64.jar -> 1687938371762, spark://10.60.24.15:37693/files/addedFile8340418444838334805org_scalactic_scalactic_2_11_3_0_4-fd6b4.jar -> 1687938371789, spark://10.60.24.15:37693/files/addedFile6368303643941656058org_apache_maven_maven_profile_2_2_1-14a13.jar -> 1687938371952, spark://10.60.24.15:37693/files/addedFile4784850045430135776org_apache_maven_maven_project_2_2_1-14a14.jar -> 1687938372109, spark://10.60.24.15:37693/files/addedFile8978999241401061738classworlds_classworlds_1_1-5675c.jar -> 1687938371859, spark://10.60.24.15:37693/files/addedFile2443633399128976327com_microsoft_sqlserver_mssql_jdbc_6_4_0_jre8-135a7.jar -> 1687938371904, spark://10.60.24.15:37693/files/addedFile8375439471574395621org_slf4j_slf4j_api_1_7_5-56248.jar -> 1687938371776, spark://10.60.24.15:37693/files/addedFile3203199928370695037org_codehaus_plexus_plexus_interpolation_1_11-19a79.jar -> 1687938371750, spark://10.60.24.15:37693/files/addedFile4356603507767992681org_apache_maven_maven_artifact_2_2_1-881cc.jar -> 1687938371975, spark://10.60.24.15:37693/files/addedFile2926898229968885619org_apache_commons_commons_lang3_3_4-2972e.jar -> 1687938371801, spark://10.60.24.15:37693/files/addedFile1181427614471513690com_nimbusds_oauth2_oidc_sdk_5_18_1-fda44.jar -> 1687938372099, spark://10.60.24.15:37693/files/addedFile790909324841831075org_apache_maven_maven_repository_metadata_2_2_1-1002b.jar -> 1687938371812, spark://10.60.24.15:37693/files/addedFile7216708682966614627org_apache_maven_maven_settings_2_2_1-fa6c6.jar -> 1687938372054, spark://10.60.24.15:37693/files/addedFile2780516324924017311org_codehaus_plexus_plexus_container_default_1_0_alpha_9_stable_1-fb79e.jar -> 1687938372010, spark://10.60.24.15:37693/files/addedFile6335364697226487582org_apache_maven_maven_plugin_registry_2_2_1-77430.jar -> 1687938371881, spark://10.60.24.15:37693/files/addedFile7685467289563692171org_apache_maven_maven_plugin_api_2_2_1-5112e.jar -> 1687938371915, spark://10.60.24.15:37693/files/addedFile3190865630717788573com_nimbusds_nimbus_jose_jwt_9_31-065b2.jar -> 1687938371964, spark://10.60.24.15:37693/files/addedFile5063058244976928872org_apache_maven_wagon_wagon_provider_api_1_0_beta_6-61efc.jar -> 1687938372089, spark://10.60.24.15:37693/files/addedFile3997909782957336268com_nimbusds_lang_tag_1_7-d5041.jar -> 1687938372077, spark://10.60.24.15:37693/files/addedFile5491777020613920757com_microsoft_azure_adal4j_1_2_0-5e9b7.jar -> 1687938371738, spark://10.60.24.15:37693/files/addedFile113100314552018668javax_mail_mail_1_4_7-d349d.jar -> 1687938371938, spark://10.60.24.15:37693/files/addedFile4291570592405472542org_apache_maven_plugins_maven_gpg_plugin_1_6-d363a.jar -> 1687938372044, spark://10.60.24.15:37693/files/addedFile7796604281496291606net_minidev_json_smart_1_3_1-475da.jar -> 1687938371847, spark://10.60.24.15:37693/files/addedFile3662280138547911438com_microsoft_azure_azure_sqldb_spark_1_0_2-2272f.jar -> 1687938372021, spark://10.60.24.15:37693/files/addedFile1212241497000342143junit_junit_3_8_1-5016d.jar -> 1687938371824, spark://10.60.24.15:37693/files/addedFile5553381464690868398com_google_code_gson_gson_2_2_4-ffff2.jar -> 1687938371719, spark://10.60.24.15:37693/files/addedFile4331226716940101919org_apache_maven_maven_artifact_manager_2_2_1-33caf.jar -> 1687938372032, spark://10.60.24.15:37693/files/addedFile4183677756918286790org_apache_commons_commons_collections4_4_1-8f9d2.jar -> 1687938371927, spark://10.60.24.15:37693/files/addedFile7202328518024188131backport_util_concurrent_backport_util_concurrent_3_1-40db4.jar -> 1687938371997, spark://10.60.24.15:37693/files/addedFile320312741712984068com_github_stephenc_jcip_jcip_annotations_1_0_1-7c837.jar -> 1687938371836) 23/06/28 07:53:30 INFO DAGScheduler: Archives for session None: Map() 23/06/28 07:53:30 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 213 (MapPartitionsRDD[470] at collect at command-1577463665195035:9) (first 15 tasks are for partitions Vector(0)) 23/06/28 07:53:30 INFO TaskSchedulerImpl: Adding task set 213.0 with 1 tasks resource profile 0 23/06/28 07:53:30 INFO FairSchedulableBuilder: Added task set TaskSet_213.0 tasks to pool 4015519563002992008 23/06/28 07:53:30 INFO TaskSetManager: Starting task 0.0 in stage 213.0 (TID 1591) (10.60.24.10, executor 0, partition 0, NODE_LOCAL, taskResourceAssignments Map()) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_147 stored as values in memory (estimated size 174.1 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO MemoryStore: Block broadcast_147_piece0 stored as bytes in memory (estimated size 41.2 KiB, free 750.6 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_147_piece0 in memory on 10.60.24.15:40507 (size: 41.2 KiB, free: 750.7 GiB) 23/06/28 07:53:30 INFO SparkContext: Created broadcast 147 from broadcast at TaskSetManager.scala:627 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_147_piece0 in memory on 10.60.24.10:41041 (size: 41.2 KiB, free: 487.7 GiB) 23/06/28 07:53:30 INFO BlockManagerInfo: Added broadcast_146_piece0 in memory on 10.60.24.10:41041 (size: 14.9 KiB, free: 487.7 GiB) 23/06/28 07:53:30 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:30.782Z","run":{"runId":"f884d48f-6d85-45c8-8872-de7a9ff7eb9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, default, LMS_lmsuu_mdl_groups_members_sql_merge]"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6872,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"MANAGED"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1687938765000,"lastAccessTime":0,"createVersion":"3.3.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_columnar_to_row","facets":{}},"inputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:30.83Z","run":{"runId":"f884d48f-6d85-45c8-8872-de7a9ff7eb9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, default, LMS_lmsuu_mdl_groups_members_sql_merge]"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6872,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"MANAGED"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1687938765000,"lastAccessTime":0,"createVersion":"3.3.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_columnar_to_row","facets":{}},"inputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:30 INFO SparkSQLExecutionContext: OpenLineage received Spark event that is configured to be skipped: SparkListenerSQLExecutionEnd 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:30.879Z","run":{"runId":"f884d48f-6d85-45c8-8872-de7a9ff7eb9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, default, LMS_lmsuu_mdl_groups_members_sql_merge]"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6872,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"MANAGED"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1687938765000,"lastAccessTime":0,"createVersion":"3.3.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_columnar_to_row","facets":{}},"inputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:30 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Configured feature flag data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load feature flag from LaunchDarkly 23/06/28 07:53:30 WARN DataClientConf: REGION environment variable is not defined. getConfForCurrentRegion will always return default value 23/06/28 07:53:30 INFO DataClientConf: Configured global ops config data source LaunchDarkly 23/06/28 07:53:30 INFO DataClientConf: Load GOC flag data from LaunchDarkly 23/06/28 07:53:31 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:30.898Z","run":{"runId":"f884d48f-6d85-45c8-8872-de7a9ff7eb9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, default, LMS_lmsuu_mdl_groups_members_sql_merge]"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6872,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"MANAGED"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1687938765000,"lastAccessTime":0,"createVersion":"3.3.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"},"spark_properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","properties":{"spark.master":"spark://10.60.24.15:7077","spark.app.name":"Databricks Shell"}},"environment-properties":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","environment-properties":{"spark.databricks.job.type":"notebook","spark.databricks.job.runId":"1","spark.databricks.clusterUsageTags.clusterName":"Databricks-Purview","spark.databricks.clusterUsageTags.azureSubscriptionId":"f242a3b6-370c-4b2e-971d-a0399c53d6f0","spark.databricks.notebook.path":"/build/includes/scala-executors/exec-sync-sqledw","mountPoints":[{"mountPoint":"/mnt/datalake-trusted","source":"abfss://trusted@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-datasets","source":"databricks-datasets"},{"mountPoint":"/mnt/co_test_raw1","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/adls-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-tracking","source":"databricks/mlflow-tracking"},{"mountPoint":"/mnt/datalake-external","source":"abfss://external@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks-results","source":"databricks-results"},{"mountPoint":"/mnt/datalake-lakehouse","source":"abfss://lakehouse@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/databricks/mlflow-registry","source":"databricks/mlflow-registry"},{"mountPoint":"/mnt/datalake-stage","source":"abfss://stage@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalake-raw/curms","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/datalakeraw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/mnt/co_test_raw","source":"abfss://raw@zedlsbi01t.dfs.core.windows.net/"},{"mountPoint":"/","source":"DatabricksRoot"},{"mountPoint":"/mnt/datalake-curated","source":"abfss://curated@zedlsbi01t.dfs.core.windows.net/"}],"spark.databricks.job.id":"982517249786434","spark.databricks.clusterUsageTags.clusterOwnerOrgId":"8119356306712193","user":"rahul.agrawal1@tafenswtest.edu.au","userId":"8162102493690770","orgId":"8119356306712193"}}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_columnar_to_row","facets":{}},"inputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:31 INFO TaskSetManager: Finished task 0.0 in stage 213.0 (TID 1591) in 684 ms on 10.60.24.10 (executor 0) (1/1) 23/06/28 07:53:31 INFO TaskSchedulerImpl: Removed TaskSet 213.0, whose tasks have all completed, from pool 4015519563002992008 23/06/28 07:53:31 INFO DAGScheduler: ResultStage 213 (collect at command-1577463665195035:9) finished in 0.686 s 23/06/28 07:53:31 INFO DAGScheduler: Job 97 is finished. Cancelling potential speculative or zombie tasks for this job 23/06/28 07:53:31 INFO TaskSchedulerImpl: Killing all running tasks in stage 213: Stage finished 23/06/28 07:53:31 INFO DAGScheduler: Job 97 finished: collect at command-1577463665195035:9, took 0.688039 s 23/06/28 07:53:31 INFO ClusterLoadMonitor: Removed query with execution ID:336. Current active queries:0 23/06/28 07:53:31 INFO SQLAppStatusListener: Recording cache-related metrics in usage logs: ioCacheWorkersDiskUsage={"10.60.24.10":{"diskUsage":90640,"lifetime":510108},"10.60.24.13":{"diskUsage":74852,"lifetime":510378}}, ioCacheNumScanTasks={"numLocalScanTasks":41,"numNonLocalScanTasks":0}, ioCacheDiskUsageLimit=1932735283200 23/06/28 07:53:31 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:31 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:31.591Z","run":{"runId":"f884d48f-6d85-45c8-8872-de7a9ff7eb9e","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.catalyst.plans.logical.Project","num-children":1,"projectList":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":"[spark_catalog, default, LMS_lmsuu_mdl_groups_members_sql_merge]"}]],"child":0},{"class":"org.apache.spark.sql.execution.datasources.LogicalRelation","num-children":0,"relation":null,"output":[[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"id","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6872,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}],[{"class":"org.apache.spark.sql.catalyst.expressions.AttributeReference","num-children":0,"name":"sql","dataType":"string","nullable":true,"metadata":{},"exprId":{"product-class":"org.apache.spark.sql.catalyst.expressions.ExprId","id":6873,"jvmId":"eb966a2f-de33-44dd-a2a0-a8970abf3f36"},"qualifier":[]}]],"catalogTable":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTable","identifier":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"lms_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"tableType":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogTableType","name":"MANAGED"},"storage":{"product-class":"org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat","locationUri":null,"inputFormat":"org.apache.hadoop.mapred.SequenceFileInputFormat","outputFormat":"org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat","serde":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","compressed":false,"properties":null},"schema":{"type":"struct","fields":[{"name":"id","type":"string","nullable":true,"metadata":{}},{"name":"sql","type":"string","nullable":true,"metadata":{}}]},"provider":"delta","partitionColumnNames":[],"owner":"root","createTime":1687938765000,"lastAccessTime":0,"createVersion":"3.3.1","properties":null,"unsupportedFeatures":[],"tracksPartitionsInCatalog":true,"schemaPreservesCase":true,"ignoredProperties":null,"tableConstraints":{"object":"scala.collection.immutable.Set$EmptySet$"},"deltaRuntimeProperties":null},"isStreaming":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_columnar_to_row","facets":{}},"inputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"schema":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SchemaDatasetFacet.json#/$defs/SchemaDatasetFacet","fields":[{"name":"id","type":"string"},{"name":"sql","type":"string"}]},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]}},"inputFacets":{}}],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:31 INFO ClusterLoadMonitor: Added query with execution ID:338. Current active queries:1 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO HiveMetaStore: 16: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:53:31 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:53:31 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:31 INFO HiveMetaStore: 2: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:31 INFO ConsoleTransport: {"eventType":"START","eventTime":"2023-06-28T07:53:31.61Z","run":{"runId":"e8c2ac05-fe4f-49fd-9b4a-0214a6fc1fae","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.DropTableCommand","num-children":0,"tableName":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"LMS_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"ifExists":true,"isView":false,"purge":false,"materialized":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_drop_table_command_warehouse_lms_lmsuu_mdl_groups_members_sql_merge","facets":{}},"inputs":[],"outputs":[{"namespace":"dbfs","name":"/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge","facets":{"dataSource":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet","name":"dbfs","uri":"dbfs"},"symlinks":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/SymlinksDatasetFacet.json#/$defs/SymlinksDatasetFacet","identifiers":[{"namespace":"/user/hive/warehouse","name":"spark_catalog.default.lms_lmsuu_mdl_groups_members_sql_merge","type":"TABLE"}]},"lifecycleStateChange":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/facets/1-0-0/LifecycleStateChangeDatasetFacet.json#/$defs/LifecycleStateChangeDatasetFacet","lifecycleStateChange":"DROP"}},"outputFacets":{}}],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:31 INFO HiveMetaStore: 17: get_database: default 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:53:31 INFO HiveMetaStore: 17: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 23/06/28 07:53:31 INFO ObjectStore: ObjectStore, initialize called 23/06/28 07:53:31 INFO ObjectStore: Initialized ObjectStore 23/06/28 07:53:31 INFO HiveMetaStore: 17: get_multi_table : db=default tbls=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=default tbls=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_database: default 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_database: default 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO ClusterLoadAvgHelper: Current cluster load: 1, Old Ema: 1.0, New Ema: 1.0 23/06/28 07:53:31 INFO HiveMetaStore: 16: drop_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=drop_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO HiveMetaStore: 16: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:31 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:32 INFO hivemetastoressimpl: deleting dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:32 INFO deprecation: org.apache.hadoop.shaded.io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 23/06/28 07:53:32 INFO NativeAzureFileSystem: FS_OP_DELETE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge] Starting delete - recursive true, don't update the last modified time for the parent folder false 23/06/28 07:53:32 WARN AzureFileSystemThreadPoolExecutor: Disabling threads for Delete operation as thread count 0 is <= 1 23/06/28 07:53:32 INFO AzureFileSystemThreadPoolExecutor: Time taken for Delete operation is: 60 ms with threads: 0 23/06/28 07:53:32 INFO NativeAzureFileSystem: FS_OP_DELETE FILE[wasbs://root@dbstorage5hyqrjw7e7tcw.blob.core.windows.net/8119356306712193/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge] Delete successful. 23/06/28 07:53:32 INFO hivemetastoressimpl: Deleted the diretory dbfs:/user/hive/warehouse/lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:32 INFO MapPartitionsRDD: Removing RDD 431 from persistence list 23/06/28 07:53:32 INFO BlockManager: Removing RDD 431 23/06/28 07:53:32 INFO ClusterLoadMonitor: Removed query with execution ID:338. Current active queries:0 23/06/28 07:53:32 INFO HiveMetaStore: 2: get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:32 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=lms_lmsuu_mdl_groups_members_sql_merge 23/06/28 07:53:32 INFO HiveMetaStore: 2: get_database: default 23/06/28 07:53:32 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:53:32 WARN DropTableCommandVisitor: Unable to find table by identifier `spark_catalog`.`default`.`LMS_lmsuu_mdl_groups_members_sql_merge` - [TABLE_OR_VIEW_NOT_FOUND] The table or view `default`.`lms_lmsuu_mdl_groups_members_sql_merge` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog. To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS. 23/06/28 07:53:32 INFO PlanUtils: isDefinedAt method failed on null 23/06/28 07:53:32 INFO ConsoleTransport: {"eventType":"COMPLETE","eventTime":"2023-06-28T07:53:32.483Z","run":{"runId":"e8c2ac05-fe4f-49fd-9b4a-0214a6fc1fae","facets":{"spark.logicalPlan":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","plan":[{"class":"org.apache.spark.sql.execution.command.DropTableCommand","num-children":0,"tableName":{"product-class":"org.apache.spark.sql.catalyst.TableIdentifier","table":"LMS_lmsuu_mdl_groups_members_sql_merge","database":"default","catalog":"spark_catalog"},"ifExists":true,"isView":false,"purge":false,"materialized":false}]},"spark_version":{"_producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","_schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet","spark-version":"3.3.1","openlineage-spark-version":"0.28.0"}}},"job":{"namespace":"adb-8119356306712193.13.#0622-000257-omphh17r","name":"adb-8119356306712193_13_azuredatabricks_net_execute_drop_table_command","facets":{}},"inputs":[],"outputs":[],"producer":"https://github.com/OpenLineage/OpenLineage/tree/0.28.0/integration/spark","schemaURL":"https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent"} 23/06/28 07:53:34 INFO ProgressReporter$: Removed result fetcher for 4015519563002992008_5380942922734008000_job-982517249786434-run-1-action-7106185942531233 23/06/28 07:53:34 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 1.0, New Ema: 0.85 23/06/28 07:53:37 INFO ClusterLoadAvgHelper: Current cluster load: 0, Old Ema: 0.85, New Ema: 0.0 23/06/28 07:53:39 INFO DriverCorral$: Cleaning the wrapper ReplId-37b9f-dc475-6cd58-8 (currently in status Idle(ReplId-37b9f-dc475-6cd58-8)) 23/06/28 07:53:39 INFO DriverCorral$: sending shutdown signal for REPL ReplId-37b9f-dc475-6cd58-8 23/06/28 07:53:39 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-37b9f-dc475-6cd58-8 23/06/28 07:53:39 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-37b9f-dc475-6cd58-8 23/06/28 07:53:39 INFO DriverCorral$: ReplId-37b9f-dc475-6cd58-8 successfully discarded 23/06/28 07:53:39 INFO DriverCorral$: Cleaning the wrapper ReplId-211eb-e6200-f0983-4 (currently in status Idle(ReplId-211eb-e6200-f0983-4)) 23/06/28 07:53:39 INFO DriverCorral$: sending shutdown signal for REPL ReplId-211eb-e6200-f0983-4 23/06/28 07:53:39 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:53:39 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:53:39 INFO ReplCrashUtils$: python shell exit code: 143 23/06/28 07:53:39 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-211eb-e6200-f0983-4 23/06/28 07:53:39 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-211eb-e6200-f0983-4 23/06/28 07:53:39 INFO DriverCorral$: ReplId-211eb-e6200-f0983-4 successfully discarded 23/06/28 07:53:46 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7983957746726698450_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:53:46 INFO ProgressReporter$: Removed result fetcher for 6095634769030761578_7909027679503007739_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:53:46 INFO ProgressReporter$: Added result fetcher for 6803957155303634069_7909027679503007739_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:53:47 INFO ProgressReporter$: Removed result fetcher for 6803957155303634069_7909027679503007739_job-980829582242690-run-5532233-action-1639381203807456 23/06/28 07:53:52 INFO DriverCorral$: Cleaning the wrapper ReplId-5e6c8-2fbdc-d9b49-5 (currently in status Idle(ReplId-5e6c8-2fbdc-d9b49-5)) 23/06/28 07:53:52 INFO DriverCorral$: sending shutdown signal for REPL ReplId-5e6c8-2fbdc-d9b49-5 23/06/28 07:53:52 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:53:52 INFO PythonDriverLocalBase$RedirectThread: Python RedirectThread exit 23/06/28 07:53:52 INFO ReplCrashUtils$: python shell exit code: 143 23/06/28 07:53:52 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-5e6c8-2fbdc-d9b49-5 23/06/28 07:53:52 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-5e6c8-2fbdc-d9b49-5 23/06/28 07:53:52 INFO DriverCorral$: ReplId-5e6c8-2fbdc-d9b49-5 successfully discarded 23/06/28 07:53:52 INFO DriverCorral$: Cleaning the wrapper ReplId-54980-b86c0-994c6-a (currently in status Idle(ReplId-54980-b86c0-994c6-a)) 23/06/28 07:53:52 INFO DriverCorral$: sending shutdown signal for REPL ReplId-54980-b86c0-994c6-a 23/06/28 07:53:52 INFO DriverCorral$: sending the interrupt signal for REPL ReplId-54980-b86c0-994c6-a 23/06/28 07:53:52 INFO DriverCorral$: waiting for localThread to stop for REPL ReplId-54980-b86c0-994c6-a 23/06/28 07:53:52 INFO DriverCorral$: ReplId-54980-b86c0-994c6-a successfully discarded 23/06/28 07:54:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:54:54 INFO DataSourceFactory$: DataSource Jdbc URL: jdbc:mariadb://consolidated-australiaeast-prod-metastore-addl-1.mysql.database.azure.com:3306/organization8119356306712193?useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertificate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt 23/06/28 07:54:54 INFO HikariDataSource: metastore-monitor - Starting... 23/06/28 07:54:54 INFO HikariDataSource: metastore-monitor - Start completed. 23/06/28 07:54:55 INFO HikariDataSource: metastore-monitor - Shutdown initiated... 23/06/28 07:54:55 INFO HikariDataSource: metastore-monitor - Shutdown completed. 23/06/28 07:54:55 INFO MetastoreMonitor: Metastore healthcheck successful (connection duration = 490 milliseconds) 23/06/28 07:55:03 INFO HiveMetaStore: 3: get_database: default 23/06/28 07:55:03 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default 23/06/28 07:55:03 ERROR CommandLineHelper$: Command [REDACTED] failed with exit code 1 out: err: 23/06/28 07:55:03 INFO DriverCorral: Metastore health check ok 23/06/28 07:55:04 INFO DriverCorral: DBFS health check ok