bug: Step Functions - An error on a map iteration does not fail the State Machine #9970
Closed
1 task done
Labels
aws:stepfunctions
AWS Step Functions
status: backlog
Triaged but not yet being worked on
status: resolved/stale
Closed due to staleness
type: bug
Bug report
Is there an existing issue for this?
Current Behavior
When running a Step function with Map-Distributed part, there are a few issues
Expected Behavior
The State machine should be with status failed
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
How are you starting localstack (e.g.,
bin/localstack
command, arguments, ordocker-compose.yml
)Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
The issue could be reproducible with the below code running with ts-node
npm install aws-sdk
npm install @types/aws-sdk --save-dev
npx ts-node /absolute/path/to/sfn-issue.ts --run-testSFN
### Code example
// Configure AWS SDK to point to LocalStack
const s3 = new AWS.S3({
region: 'us-west-2',
endpoint: 'http://localhost:4566', // LocalStack endpoint
accessKeyId: 'test', // Dummy credentials for LocalStack
secretAccessKey: 'test',
s3ForcePathStyle: true, // required for localstack
s3BucketEndpoint: false,
sslEnabled: false
});
const stepFunctions = new AWS.StepFunctions({
endpoint: 'http://localhost:4566', // Default LocalStack endpoint
region: 'us-east-1', // Your preferred region
accessKeyId: 'test', // Dummy credentials for LocalStack
secretAccessKey: 'test'
});
async function writeToS3(bucketName: string, objectPath: string, objectData: string|Buffer) {
const params: AWS.S3.Types.PutObjectRequest = {
Bucket: bucketName,
Key: objectPath,
Body: objectData,
ContentType: 'application/json'
};
try {
await s3.createBucket({ Bucket: bucketName }).promise();
} catch (error) {
console.log(error);
}
}
const sfn = {
StartAt: 'MapDistributed',
States: {
MapDistributed: {
Type: 'Map',
Parameters: {
'variable.$': '$$.Map.Item.Value'
},
ItemReader: {
ReaderConfig: {
InputType: 'JSON'
},
Resource: 'arn:aws:states:::s3:getObject',
Parameters: {
Bucket: 'some-bucket-test',
Key: 'someKey'
}
},
ItemProcessor: {
ProcessorConfig: {
Mode: 'DISTRIBUTED',
ExecutionType: 'STANDARD'
},
StartAt: 'Run an SFN',
States: {
'Run an SFN': {
Type: 'Task',
Resource: 'arn:aws:states:::states:startExecution.sync:2',
Parameters: {
StateMachineArn: 'some-state-machine-arn',
Name: 'somename',
Input: {
'variable.$': '$.variable'
}
},
ResultPath: null,
End: true
}
}
},
End: true
}
}
};
async function testSFN() {
const params = {
name: 'arn:aws:states:us-east-1:000000000000:stateMachine:test',
definition: JSON.stringify(sfn),
roleArn: 'arn:aws:iam::123456789012:role/DummyRole'
};
const stateMachine = await stepFunctions.createStateMachine(params).promise();
await writeToS3('some-bucket-test', 'someKey', JSON.stringify(['test123']));
}
if (process.argv.includes('--run-testSFN')) {
testSFN().catch(error => {
console.error(error);
process.exit(1);
});
}
Environment
Anything else?
### The error appears on localstack-main logs
localstack-main | 2024-01-02T09:53:42.511 INFO --- [ asgi_gw_3] localstack.request.aws : AWS stepfunctions.StartExecution => 200
localstack-main | 2024-01-02T09:53:42.542 INFO --- [ asgi_gw_2] localstack.request.aws : AWS stepfunctions.DescribeExecution => 200
localstack-main | 2024-01-02T09:53:42.659 ERROR --- [15628 (eval)] l.s.s.a.c.eval_component : Exception=FailureEventException, Error=StepFunctions.StateMachineDoesNotExistException, Details={"taskFailedEventDetails": {"error": "StepFunctions.StateMachineDoesNotExistException", "cause": "State Machine Does Not Exist: 'some-state-machine-arn' (Service: AWSStepFunctions; Status Code: 400; Error Code: StateMachineDoesNotExist; Request ID: 47a614f1-80ad-4462-96d5-ba7490405ff1; Proxy: null)", "resource": "startExecution.sync:2", "resourceType": "states"}} at '(StateTaskServiceSfn| {'comment': None, 'input_path': (InputPath| {'input_path_src': '$'}, 'output_path': (OutputPath| {'output_path': '$'}, 'state_entered_event_type': 'TaskStateEntered', 'state_exited_event_type': 'TaskStateExited', 'result_path': (ResultPath| {'result_path_src': None}, 'result_selector': None, 'retry': None, 'catch': None, 'timeout': (TimeoutSeconds| {'timeout_seconds': 99999999, 'is_default': None}, 'heartbeat': None, 'parameters': (Parameters| {'payload_tmpl': (PayloadTmpl| {'payload_bindings': [(PayloadBindingValue| {'field': 'StateMachineArn', 'value': (PayloadValueStr| {'val': 'some-state-machine-arn'}}, (PayloadBindingValue| {'field': 'Name', 'value': (PayloadValueStr| {'val': 'somename'}}, (PayloadBindingValue| {'field': 'Input', 'value': (PayloadTmpl| {'payload_bindings': [(PayloadBindingPath| {'field': 'variable', 'path': '$.variable'}]}}]}}, 'name': 'Run an SFN', 'state_type': <StateType.Task: 15>, 'continue_with': <localstack.services.stepfunctions.asl.component.state.state_continue_with.ContinueWithEnd object at 0xfffedf02f110>, 'resource': (ServiceResource| {'_region': '', '_account': '', 'resource_arn': 'arn:aws:states:::states:startExecution.sync:2', 'partition': 'aws', 'service_name': 'states', 'api_name': 'states', 'api_action': 'startExecution', 'condition': 'sync:2'}}'
localstack-main | 2024-01-02T09:53:47.572 INFO --- [ asgi_gw_0] localstack.request.aws : AWS stepfunctions.DescribeExecution => 200
localstack-main | 2024-01-02T09:53:47.601 INFO --- [ asgi_gw_3] localstack.request.aws : AWS stepfunctions.GetExecutionHistory => 200
### No error when calling describe execution
[
{
timestamp: 2024-01-02T09:53:42.511Z,
type: 'ExecutionStarted',
id: 1,
previousEventId: 0,
executionStartedEventDetails: {
input: '{}',
inputDetails: [Object],
roleArn: 'arn:aws:iam::123456789012:role/DummyRole'
}
},
{
timestamp: 2024-01-02T09:53:42.514Z,
type: 'MapStateEntered',
id: 2,
previousEventId: 0,
stateEnteredEventDetails: { name: 'MapDistributed', input: '{}', inputDetails: [Object] }
},
{
timestamp: 2024-01-02T09:53:42.515Z,
type: 'MapStateStarted',
id: 3,
previousEventId: 2,
mapStateStartedEventDetails: { length: 0 }
},
{
timestamp: 2024-01-02T09:53:42.516Z,
type: 'MapRunStarted',
id: 4,
previousEventId: 3,
mapRunStartedEventDetails: {
mapRunArn: 'arn:aws:states:us-east-1:000000000000:mapRun:test/68df36fd-ea01-4621-a4a6-46b33d64bf81:99d6d353-15d5-4669-8244-0f5dc7013904'
}
},
{
timestamp: 2024-01-02T09:53:42.660Z,
type: 'MapRunSucceeded',
id: 5,
previousEventId: 4
},
{
timestamp: 2024-01-02T09:53:42.660Z,
type: 'MapStateSucceeded',
id: 6,
previousEventId: 5
},
{
timestamp: 2024-01-02T09:53:42.683Z,
type: 'MapStateExited',
id: 7,
previousEventId: 5,
stateExitedEventDetails: {
name: 'MapDistributed',
output: '[null]',
outputDetails: [Object]
}
},
{
timestamp: 2024-01-02T09:53:42.683Z,
type: 'ExecutionSucceeded',
id: 8,
previousEventId: 7,
executionSucceededEventDetails: { output: '[null]', outputDetails: [Object] }
}
]
The text was updated successfully, but these errors were encountered: