Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renaming process to scenario - backend #1911

Merged
merged 2 commits into from
Jul 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Nussknacker versions
* [#1357](https://github.com/TouK/nussknacker/pull/1357) Add run mode to nodes to be able to determine if we are inside e.g. test process run.
Run mode is can be declared as a dependency in generic node transformations. Nodes created via `@MethodToInvoke` can declare `RunMode` as an implicit parameter.
`RunMode` is also available in `FlinkCustomNodeContext`.
* [#1911](https://github.com/TouK/nussknacker/pull/1911) Rename `process` to `scenario`, `subprocess` to `fragment` in messages at backend and some test cases names

0.3.1 (not released yet)
------------------------
Expand Down
7 changes: 7 additions & 0 deletions docs/components/ComponentConfiguration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Component configuration
=======================

```

```

Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ sealed trait CustomActionError extends Exception {
case class CustomActionFailure(request: CustomActionRequest, msg: String) extends CustomActionError

case class CustomActionInvalidStatus(request: CustomActionRequest, stateStatusName: String) extends CustomActionError {
override val msg: String = s"Process status: $stateStatusName is not allowed for action ${request.name}"
override val msg: String = s"Scenario status: $stateStatusName is not allowed for action ${request.name}"
}

case class CustomActionNotImplemented(request: CustomActionRequest) extends CustomActionError {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,31 +44,31 @@ object SimpleProcessStateDefinitionManager extends ProcessStateDefinitionManager
)

val statusTooltipsMap: Map[StateStatus, String] = Map(
SimpleStateStatus.FailedToGet -> "There are problems obtaining the process state. Please check if your engine is working properly.",
SimpleStateStatus.Unknown -> "Unknown state of the process. We can't recognize process state.",
SimpleStateStatus.NotDeployed -> "The process is not deployed.",
SimpleStateStatus.DuringDeploy -> "The process has been already started and currently is being deployed.",
SimpleStateStatus.Running -> "The process has been successfully deployed and currently is running.",
SimpleStateStatus.Canceled -> "The process has been successfully cancelled.",
SimpleStateStatus.DuringCancel -> "The process currently is being canceled.",
SimpleStateStatus.Failed -> "There are some problems with process.",
SimpleStateStatus.Finished -> "The process completed successfully.",
SimpleStateStatus.Error -> "There are some errors. Please check if everything is okay with process!",
SimpleStateStatus.Warning -> "There are some warnings. Please check if everything is okay with process!"
SimpleStateStatus.FailedToGet -> "There are problems obtaining the scenario state. Please check if your engine is working properly.",
SimpleStateStatus.Unknown -> "Unknown state of the scenario. We can't recognize scenario state.",
SimpleStateStatus.NotDeployed -> "The scenario is not deployed.",
SimpleStateStatus.DuringDeploy -> "The scenario has been already started and currently is being deployed.",
SimpleStateStatus.Running -> "The scenario has been successfully deployed and currently is running.",
SimpleStateStatus.Canceled -> "The scenario has been successfully cancelled.",
SimpleStateStatus.DuringCancel -> "The scenario currently is being canceled.",
SimpleStateStatus.Failed -> "There are some problems with scenario.",
SimpleStateStatus.Finished -> "The scenario completed successfully.",
SimpleStateStatus.Error -> "There are some errors. Please check if everything is okay with scenario!",
SimpleStateStatus.Warning -> "There are some warnings. Please check if everything is okay with scenario!"
)

val statusDescriptionsMap: Map[StateStatus, String] = Map(
SimpleStateStatus.FailedToGet -> "Failed to get a state of the process.",
SimpleStateStatus.Unknown -> "Unknown state of the process.",
SimpleStateStatus.NotDeployed -> "The process is not deployed.",
SimpleStateStatus.DuringDeploy -> "The process is being deployed.",
SimpleStateStatus.Running -> "The process is running.",
SimpleStateStatus.Canceled -> "The process is canceled.",
SimpleStateStatus.DuringCancel -> "The process is being canceled.",
SimpleStateStatus.Failed -> "There are some problems with process.",
SimpleStateStatus.Finished -> "The process has finished.",
SimpleStateStatus.Error -> "There are errors establishing a process state.",
SimpleStateStatus.Warning -> "There are some warnings establishing a process state."
SimpleStateStatus.FailedToGet -> "Failed to get a state of the scenario.",
SimpleStateStatus.Unknown -> "Unknown state of the scenario.",
SimpleStateStatus.NotDeployed -> "The scenario is not deployed.",
SimpleStateStatus.DuringDeploy -> "The scenario is being deployed.",
SimpleStateStatus.Running -> "The scenario is running.",
SimpleStateStatus.Canceled -> "The scenario is canceled.",
SimpleStateStatus.DuringCancel -> "The scenario is being canceled.",
SimpleStateStatus.Failed -> "There are some problems with scenario.",
SimpleStateStatus.Finished -> "The scenario has finished.",
SimpleStateStatus.Error -> "There are errors establishing a scenario state.",
SimpleStateStatus.Warning -> "There are some warnings establishing a scenario state."
)

override def statusIcon(stateStatus: StateStatus): Option[URI] =
Expand All @@ -89,21 +89,21 @@ object SimpleProcessStateDefinitionManager extends ProcessStateDefinitionManager
statusDescriptionsMap.get(stateStatus)

def shouldBeRunningTooltip(deployedVersionId: Long, user: String): String =
s"Process deployed in version ${deployedVersionId} (by ${user}), should be running!"
s"Scenario deployed in version ${deployedVersionId} (by ${user}), should be running!"

def mismatchDeployedVersionTooltip(deployedVersionId: Long, exceptedVersionId: Long, user: String): String =
s"Process deployed in version ${deployedVersionId} (by ${user}), expected version ${exceptedVersionId}!"
s"Scenario deployed in version ${deployedVersionId} (by ${user}), expected version ${exceptedVersionId}!"

def missingDeployedVersionTooltip(exceptedVersionId: Long, user: String): String =
s"Process deployed without version (by ${user}), expected version ${exceptedVersionId}!"
s"Scenario deployed without version (by ${user}), expected version ${exceptedVersionId}!"

val shouldBeRunningDescription: String = "Process currently is not running!"
val shouldBeRunningDescription: String = "Scenario currently is not running!"

val mismatchDeployedVersionDescription: String = "Deployed process mismatch version!"
val mismatchDeployedVersionDescription: String = "Deployed scenario mismatch version!"

val missingDeployedVersionDescription: String = "Missing version of deployed process!"
val missingDeployedVersionDescription: String = "Missing version of deployed scenario!"

val processWithoutActionMessage: String = "Process state error - no actions found!"
val processWithoutActionMessage: String = "Scenario state error - no actions found!"

val deployFailedIcon: URI = URI.create("/assets/states/deploy-failed.svg")

Expand All @@ -114,8 +114,8 @@ object SimpleProcessStateDefinitionManager extends ProcessStateDefinitionManager
val notDeployedWarningIcon: URI = URI.create("/assets/states/not-deployed-warning.svg")

def shouldNotBeRunningMessage(deployed: Boolean): String =
if (deployed) "Process has been canceled but still is running!"
else "Process has been never deployed but now is running!"
if (deployed) "Scenario has been canceled but still is running!"
else "Scenario has been never deployed but now is running!"

def shouldNotBeRunningIcon(deployed: Boolean): URI =
if (deployed) stoppingWarningIcon else notDeployedWarningIcon
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ class SimpleProcessStateSpec extends FunSpec with Matchers with Inside with Eith
def createProcessState(stateStatus: StateStatus): ProcessState =
SimpleProcessState(ExternalDeploymentId("12"), stateStatus)

it ("process state should be during deploy") {
it ("scenario state should be during deploy") {
val state = createProcessState(SimpleStateStatus.DuringDeploy)
state.status.isDuringDeploy shouldBe true
state.allowedActions shouldBe List(ProcessActionType.Cancel)
}

it ("process state should be running") {
it ("scenario state should be running") {
val state = createProcessState(SimpleStateStatus.Running)
state.status.isRunning shouldBe true
state.allowedActions shouldBe List(ProcessActionType.Cancel, ProcessActionType.Pause, ProcessActionType.Deploy)
}

it ("process state should be finished") {
it ("scenario state should be finished") {
val state = createProcessState(SimpleStateStatus.Finished)
state.status.isFinished shouldBe true
state.allowedActions shouldBe List(ProcessActionType.Deploy, ProcessActionType.Archive)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ class GenericItSpec extends FunSuite with FlinkSpec with Matchers with KafkaSpec
.sink("end", "#outPutVar", "kafka-json", "topic" -> s"'$topicOut'")
))

logger.info("Starting union process")
logger.info("Starting union scenario")
run(process) {
logger.info("Waiting for consumer")
val consumer = kafkaClient.createConsumer().consume(topicOut, secondsToWaitForAvro)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@ class DeploymentActor(service: PeriodicProcessService, interval: FiniteDuration)

override def receive: Receive = {
case CheckToBeDeployed =>
logger.debug("Checking processes to be deployed")
logger.debug("Checking scenarios to be deployed")
service.findToBeDeployed.foreach { runDetails => self ! WaitingForDeployment(runDetails.toList) }
case WaitingForDeployment(Nil) =>
case WaitingForDeployment(runDetails :: _) =>
logger.info("Found a process to be deployed: {}", runDetails.id)
logger.info("Found a scenario to be deployed: {}", runDetails.id)
service.deploy(runDetails) onComplete { result => self ! DeploymentCompleted(result.isSuccess) }
context.become(ongoingDeployment(runDetails))
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ class PeriodicProcessManager(val delegate: ProcessManager,
case (_: GraphProcess, Left(error)) =>
Future.failed(new PeriodicProcessException(error))
case _ =>
Future.failed(new PeriodicProcessException("Only periodic processes can be scheduled"))
Future.failed(new PeriodicProcessException("Only periodic scenarios can be scheduled"))
}
}

Expand All @@ -93,7 +93,7 @@ class PeriodicProcessManager(val delegate: ProcessManager,
.map(_.isDefined)
.flatMap(shouldStop => {
if (shouldStop) {
logger.info(s"Process ${processVersion.processName} is running or scheduled. Cancelling before reschedule")
logger.info(s"Scenario ${processVersion.processName} is running or scheduled. Cancelling before reschedule")
cancel(processVersion.processName, user).map(_ => ())
}
else Future.successful(())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ class PeriodicProcessManagerProvider(delegate: ProcessManagerProvider,
override def name: String = s"${delegate.name}Periodic"

override def createProcessManager(modelData: ModelData, config: Config): ProcessManager = {
logger.info("Creating periodic process manager")
logger.info("Creating periodic scenario manager")
val delegateProcessManager = delegate.createProcessManager(modelData, config)

import net.ceedubs.ficus.Ficus._
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ class PeriodicProcessService(delegateProcessManager: ProcessManager,
case Right(scheduleDates) if scheduleDates.forall(_._2.isEmpty) =>
Future.failed(new PeriodicProcessException(s"No future date determined by $schedule"))
case Right(scheduleDates) =>
logger.info("Scheduling periodic process: {} on {}", processVersion, scheduleDates)
logger.info("Scheduling periodic scenario: {} on {}", processVersion, scheduleDates)
jarManager.prepareDeploymentWithJar(processVersion, processJson).flatMap { deploymentWithJarData =>
initialSchedule(schedule, scheduleDates, deploymentWithJarData)
}
Expand Down Expand Up @@ -93,7 +93,7 @@ class PeriodicProcessService(delegateProcessManager: ProcessManager,
private def checkIfNotRunning(toDeploy: PeriodicProcessDeployment): Future[Option[PeriodicProcessDeployment]] = {
delegateProcessManager.findJobStatus(toDeploy.periodicProcess.processVersion.processName).map {
case Some(state) if state.isDeployed =>
logger.debug(s"Deferring run of ${toDeploy.display} as process is currently running")
logger.debug(s"Deferring run of ${toDeploy.display} as scenario is currently running")
None
case _ => Some(toDeploy)
}
Expand Down Expand Up @@ -212,20 +212,20 @@ class PeriodicProcessService(delegateProcessManager: ProcessManager,
additionalDeploymentDataProvider.prepareAdditionalData(deployment))
val deploymentWithJarData = deployment.periodicProcess.deploymentData
val deploymentAction = for {
_ <- Future.successful(logger.info("Deploying process {} for deployment id {}", deploymentWithJarData.processVersion, id))
_ <- Future.successful(logger.info("Deploying scenario {} for deployment id {}", deploymentWithJarData.processVersion, id))
externalDeploymentId <- jarManager.deployWithJar(deploymentWithJarData, deploymentData)
} yield externalDeploymentId
deploymentAction
.flatMap { externalDeploymentId =>
logger.info("Process has been deployed {} for deployment id {}", deploymentWithJarData.processVersion, id)
logger.info("Scenario has been deployed {} for deployment id {}", deploymentWithJarData.processVersion, id)
//TODO: add externalDeploymentId??
scheduledProcessesRepository.markDeployed(id)
.flatMap(_ => scheduledProcessesRepository.findProcessData(id))
.flatMap(afterChange => handleEvent(DeployedEvent(afterChange, externalDeploymentId))).run
}
// We can recover since deployment actor watches only future completion.
.recoverWith { case exception =>
logger.error(s"Process deployment ${deployment.display} failed", exception)
logger.error(s"Scenario deployment ${deployment.display} failed", exception)
markFailedAction(deployment, None).run
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ class RescheduleFinishedActor(service: PeriodicProcessService, interval: FiniteD

override def receive: Receive = {
case CheckStates =>
logger.debug("Checking processes to be rescheduled or marked as failed")
logger.debug("Checking scenarios to be rescheduled or marked as failed")
service.handleFinished
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ object SchedulePropertyExtractor {
processDeploymentData match {
case GraphProcess(processAsJson) =>
for {
canonicalProcess <- ProcessMarshaller.fromJson(processAsJson).leftMap(_ => "Process is unparseable").toEither.right
canonicalProcess <- ProcessMarshaller.fromJson(processAsJson).leftMap(_ => "Scenario is unparseable").toEither.right
property <- canonicalProcess.metaData.additionalFields.flatMap(_.properties.get(name)).toRight(s"$name property is missing").right
} yield property
case CustomProcess(_) => Left("Custom process is not supported")
case CustomProcess(_) => Left("Custom scenario is not supported")
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ private[periodic] class FlinkJarManager(flinkClient: FlinkClient,

override def prepareDeploymentWithJar(processVersion: ProcessVersion,
processJson: String): Future[DeploymentWithJarData] = {
logger.info(s"Prepare deployment for process: $processVersion")
logger.info(s"Prepare deployment for scenario: $processVersion")
copyJarToLocalDir(processVersion).flatMap { jarFileName =>
val deploymentWithJarData = model.DeploymentWithJarData(
processVersion = processVersion,
Expand All @@ -68,7 +68,7 @@ private[periodic] class FlinkJarManager(flinkClient: FlinkClient,

override def deployWithJar(deploymentWithJarData: DeploymentWithJarData, deploymentData: DeploymentData): Future[Option[ExternalDeploymentId]] = {
val processVersion = deploymentWithJarData.processVersion
logger.info(s"Deploying process ${processVersion.processName.value}, version id: ${processVersion.versionId} and jar: ${deploymentWithJarData.jarFileName}")
logger.info(s"Deploying scenario ${processVersion.processName.value}, version id: ${processVersion.versionId} and jar: ${deploymentWithJarData.jarFileName}")
val jarFile = jarsDir.resolve(deploymentWithJarData.jarFileName).toFile
val args = FlinkProcessManager.prepareProgramArgs(deploymentWithJarData.modelConfig,
processVersion,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ class CronSchedulePropertyExtractorTest extends FunSuite

private val extractor = CronSchedulePropertyExtractor()

test("should fail for custom process") {
test("should fail for custom scenario") {
val result = extractor(CustomProcess("test"))

inside(result) { case Left("Custom process is not supported") => }
inside(result) { case Left("Custom scenario is not supported") => }
}

test("should fail for unparseable process json") {
test("should fail for unparseable scenario json") {
val result = extractor(GraphProcess("broken"))

inside(result) { case Left("Process is unparseable") => }
inside(result) { case Left("Scenario is unparseable") => }
}

test("should fail for missing cron property") {
Expand Down
Loading