[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Step Functions - An error on a map iteration does not fail the State Machine #9970

Closed
1 task done
amithi-cider opened this issue Jan 2, 2024 · 2 comments
Closed
1 task done
Assignees
Labels
aws:stepfunctions AWS Step Functions status: backlog Triaged but not yet being worked on status: resolved/stale Closed due to staleness type: bug Bug report

Comments

@amithi-cider
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When running a Step function with Map-Distributed part, there are a few issues

  1. Errors from within a map iteration does not appear when describing the execution or the map run - seems like the Parameters does not work, only ItemSelector https://docs.aws.amazon.com/step-functions/latest/dg/input-output-itemselector.html
  2. Even after changing from Parameters to ItemSelector, when trying to run a state machine that does not exist, the SFN still appears as successful - output is [null] - no error in the output as well.

Expected Behavior

The State machine should be with status failed

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

using the docker-compose.yml with docker-compose up

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

only using aws sdk on nodejs.

The issue could be reproducible with the below code running with ts-node
npm install aws-sdk
npm install @types/aws-sdk --save-dev
npx ts-node /absolute/path/to/sfn-issue.ts --run-testSFN

### Code example

// Configure AWS SDK to point to LocalStack
const s3 = new AWS.S3({
region: 'us-west-2',
endpoint: 'http://localhost:4566', // LocalStack endpoint
accessKeyId: 'test', // Dummy credentials for LocalStack
secretAccessKey: 'test',
s3ForcePathStyle: true, // required for localstack
s3BucketEndpoint: false,
sslEnabled: false
});
const stepFunctions = new AWS.StepFunctions({
endpoint: 'http://localhost:4566', // Default LocalStack endpoint
region: 'us-east-1', // Your preferred region
accessKeyId: 'test', // Dummy credentials for LocalStack
secretAccessKey: 'test'
});
async function writeToS3(bucketName: string, objectPath: string, objectData: string|Buffer) {
const params: AWS.S3.Types.PutObjectRequest = {
Bucket: bucketName,
Key: objectPath,
Body: objectData,
ContentType: 'application/json'
};
try {
await s3.createBucket({ Bucket: bucketName }).promise();
} catch (error) {
console.log(error);
}

await s3.putObject(params).promise();

}
const sfn = {
StartAt: 'MapDistributed',
States: {
MapDistributed: {
Type: 'Map',
Parameters: {
'variable.$': '$$.Map.Item.Value'
},
ItemReader: {
ReaderConfig: {
InputType: 'JSON'
},
Resource: 'arn:aws:states:::s3:getObject',
Parameters: {
Bucket: 'some-bucket-test',
Key: 'someKey'
}
},
ItemProcessor: {
ProcessorConfig: {
Mode: 'DISTRIBUTED',
ExecutionType: 'STANDARD'
},
StartAt: 'Run an SFN',
States: {
'Run an SFN': {
Type: 'Task',
Resource: 'arn:aws:states:::states:startExecution.sync:2',
Parameters: {
StateMachineArn: 'some-state-machine-arn',
Name: 'somename',
Input: {
'variable.$': '$.variable'
}
},
ResultPath: null,
End: true
}
}
},
End: true
}
}
};

async function testSFN() {
const params = {
name: 'arn:aws:states:us-east-1:000000000000:stateMachine:test',
definition: JSON.stringify(sfn),
roleArn: 'arn:aws:iam::123456789012:role/DummyRole'
};
const stateMachine = await stepFunctions.createStateMachine(params).promise();
await writeToS3('some-bucket-test', 'someKey', JSON.stringify(['test123']));

const executionData = await stepFunctions.startExecution({
    stateMachineArn: stateMachine.stateMachineArn
}).promise();
await new Promise(resolve => setTimeout(resolve, 10000));
console.log(await stepFunctions.describeExecution({ executionArn: executionData.executionArn }).promise());

}

if (process.argv.includes('--run-testSFN')) {
testSFN().catch(error => {
console.error(error);
process.exit(1);
});
}

Environment

- OS: Apple M1 Pro MacOS 14.1.1 (23B81)
- LocalStack: latest (3.0.2)

Anything else?

### The error appears on localstack-main logs

localstack-main | 2024-01-02T09:53:42.511 INFO --- [ asgi_gw_3] localstack.request.aws : AWS stepfunctions.StartExecution => 200
localstack-main | 2024-01-02T09:53:42.542 INFO --- [ asgi_gw_2] localstack.request.aws : AWS stepfunctions.DescribeExecution => 200
localstack-main | 2024-01-02T09:53:42.659 ERROR --- [15628 (eval)] l.s.s.a.c.eval_component : Exception=FailureEventException, Error=StepFunctions.StateMachineDoesNotExistException, Details={"taskFailedEventDetails": {"error": "StepFunctions.StateMachineDoesNotExistException", "cause": "State Machine Does Not Exist: 'some-state-machine-arn' (Service: AWSStepFunctions; Status Code: 400; Error Code: StateMachineDoesNotExist; Request ID: 47a614f1-80ad-4462-96d5-ba7490405ff1; Proxy: null)", "resource": "startExecution.sync:2", "resourceType": "states"}} at '(StateTaskServiceSfn| {'comment': None, 'input_path': (InputPath| {'input_path_src': '$'}, 'output_path': (OutputPath| {'output_path': '$'}, 'state_entered_event_type': 'TaskStateEntered', 'state_exited_event_type': 'TaskStateExited', 'result_path': (ResultPath| {'result_path_src': None}, 'result_selector': None, 'retry': None, 'catch': None, 'timeout': (TimeoutSeconds| {'timeout_seconds': 99999999, 'is_default': None}, 'heartbeat': None, 'parameters': (Parameters| {'payload_tmpl': (PayloadTmpl| {'payload_bindings': [(PayloadBindingValue| {'field': 'StateMachineArn', 'value': (PayloadValueStr| {'val': 'some-state-machine-arn'}}, (PayloadBindingValue| {'field': 'Name', 'value': (PayloadValueStr| {'val': 'somename'}}, (PayloadBindingValue| {'field': 'Input', 'value': (PayloadTmpl| {'payload_bindings': [(PayloadBindingPath| {'field': 'variable', 'path': '$.variable'}]}}]}}, 'name': 'Run an SFN', 'state_type': <StateType.Task: 15>, 'continue_with': <localstack.services.stepfunctions.asl.component.state.state_continue_with.ContinueWithEnd object at 0xfffedf02f110>, 'resource': (ServiceResource| {'_region': '', '_account': '', 'resource_arn': 'arn:aws:states:::states:startExecution.sync:2', 'partition': 'aws', 'service_name': 'states', 'api_name': 'states', 'api_action': 'startExecution', 'condition': 'sync:2'}}'
localstack-main | 2024-01-02T09:53:47.572 INFO --- [ asgi_gw_0] localstack.request.aws : AWS stepfunctions.DescribeExecution => 200
localstack-main | 2024-01-02T09:53:47.601 INFO --- [ asgi_gw_3] localstack.request.aws : AWS stepfunctions.GetExecutionHistory => 200

### No error when calling describe execution

[
{
timestamp: 2024-01-02T09:53:42.511Z,
type: 'ExecutionStarted',
id: 1,
previousEventId: 0,
executionStartedEventDetails: {
input: '{}',
inputDetails: [Object],
roleArn: 'arn:aws:iam::123456789012:role/DummyRole'
}
},
{
timestamp: 2024-01-02T09:53:42.514Z,
type: 'MapStateEntered',
id: 2,
previousEventId: 0,
stateEnteredEventDetails: { name: 'MapDistributed', input: '{}', inputDetails: [Object] }
},
{
timestamp: 2024-01-02T09:53:42.515Z,
type: 'MapStateStarted',
id: 3,
previousEventId: 2,
mapStateStartedEventDetails: { length: 0 }
},
{
timestamp: 2024-01-02T09:53:42.516Z,
type: 'MapRunStarted',
id: 4,
previousEventId: 3,
mapRunStartedEventDetails: {
mapRunArn: 'arn:aws:states:us-east-1:000000000000:mapRun:test/68df36fd-ea01-4621-a4a6-46b33d64bf81:99d6d353-15d5-4669-8244-0f5dc7013904'
}
},
{
timestamp: 2024-01-02T09:53:42.660Z,
type: 'MapRunSucceeded',
id: 5,
previousEventId: 4
},
{
timestamp: 2024-01-02T09:53:42.660Z,
type: 'MapStateSucceeded',
id: 6,
previousEventId: 5
},
{
timestamp: 2024-01-02T09:53:42.683Z,
type: 'MapStateExited',
id: 7,
previousEventId: 5,
stateExitedEventDetails: {
name: 'MapDistributed',
output: '[null]',
outputDetails: [Object]
}
},
{
timestamp: 2024-01-02T09:53:42.683Z,
type: 'ExecutionSucceeded',
id: 8,
previousEventId: 7,
executionSucceededEventDetails: { output: '[null]', outputDetails: [Object] }
}
]

@amithi-cider amithi-cider added status: triage needed Requires evaluation by maintainers type: bug Bug report labels Jan 2, 2024
@localstack-bot
Copy link
Collaborator

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.

@MEPalma MEPalma self-assigned this Jan 2, 2024
@MarcelStranak MarcelStranak added aws:stepfunctions AWS Step Functions status: backlog Triaged but not yet being worked on and removed status: triage needed Requires evaluation by maintainers labels Jan 4, 2024
@localstack-bot
Copy link
Collaborator

Hello 👋! It looks like this issue hasn’t been active in longer than five months. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

@localstack-bot localstack-bot added the status: stale To be closed soon due to staleness label Jun 2, 2024
@localstack-bot localstack-bot added status: resolved/stale Closed due to staleness and removed status: stale To be closed soon due to staleness labels Jun 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws:stepfunctions AWS Step Functions status: backlog Triaged but not yet being worked on status: resolved/stale Closed due to staleness type: bug Bug report
Projects
None yet
Development

No branches or pull requests

4 participants