[go: nahoru, domu]

Skip to content

A Singer tap for AWS Athena. Live coded using the Meltano SDK: https://youtu.be/Mg0wVuWGtgI

Notifications You must be signed in to change notification settings

freimer/tap-athena

 
 

Repository files navigation

tap-athena

tap-athena is a Singer tap for Athena.

Built with the Meltano Tap SDK for Singer Taps.

Installation

pipx install git+https://github.com/MeltanoLabs/tap-athena.git

Configuration

Accepted Config Options

  • aws_access_key_id
  • aws_secret_access_key
  • s3_staging_dir
  • schema_name
  • aws_region

A full list of supported settings and capabilities for this tap is available by running:

tap-athena --about

Source Authentication and Authorization

Authentication is performed using AWS credentials, as provided from config settings descried above.

Usage

You can easily run tap-athena by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-athena --version
tap-athena --help
tap-athena --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the tap_athena/tests subfolder and then run:

poetry run pytest

You can also test the tap-athena CLI interface directly using poetry run:

poetry run tap-athena --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Open the meltano.yml and follow any "TODO" items listed in the file.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-athena
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-athena --version
# OR run a test `elt` pipeline:
meltano elt tap-athena target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.

About

A Singer tap for AWS Athena. Live coded using the Meltano SDK: https://youtu.be/Mg0wVuWGtgI

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 87.4%
  • Shell 12.6%