LibreFOIA is a comprehensive tool designed to handle various tasks related to email processing, data extraction, and database management.

This repository contains scripts and configurations for reading emails, extracting data from different file types, storing information in a database, and sending emails.

• Installation
• Usage
  • Reading Emails
  • Adding Emails to Database
  • Sending Emails
  • Bulk Email Sender
  • Finding Emails Not in Database
• Scripts
  • config/emailService.js
  • config/models.js
  • data_extractor
• Contributing
• License

To install the necessary dependencies, run:

npm install

You need to setup a Google Service Account https://cloud.google.com/iam/docs/service-accounts-create

Create a new Project

https://console.cloud.google.com/home/dashboard?project=autocereri544&authuser=8

To get the permissions that you need to create service accounts, ask your administrator to grant you the Create Service Accounts (roles/iam.serviceAccountCreator) IAM role on the project.

Download the json file that you will generate and add it to ./src/config/service_account.json

Navigate to the src folder

cd ./src

To send an email:

node step_1_0_send_email.js --from <email_address> --to <email_address> --subject "Subject" --body "Body of the email"

To send emails in bulk from a JSON file:

node step_1_1_bulkEmailSender.js --file <path_to_json_file> --subject "Subject" --body <path_to_body_file> --from <from_email>

To read emails from a specified email address:

node step_3_0_read_emails.js --email <email_address> --max <number_of_emails> --download

To read emails from a specified email address and save to a file:

node step_3_0_read_emails.js --email <email_address> --max <number_of_emails> --download

To add the fetched emails to the database:

curl -X POST -H "Content-Type: application/json" --data-binary "@response_read_emails.json" http://localhost:3000/add-emails

To find emails that are not in the database but are present in a JSON file of emails you have send:

node step_2_2_identify_unanswered_emails_by_subject.js --subject "Subject" --file <path_to_json_file>

Handles the creation of sent email records.

Defines the Sequelize models for interacting with the database.

Contains utilities for extracting data from various file types:

• csvUtils.js: Extracts text from CSV files.
• data_extractor.js: Main data extraction logic.
• db_setup.js: Sets up the database connection.
• excelUtils.js: Extracts text from Excel files.
• fileUtils.js: Utilities for file extraction.
• pdfUtils.js: Extracts text from PDF files.
• wordUtils.js: Extracts text from Word and ODT files.

Contributions are welcome! Please open an issue or submit a pull request with your changes.

This project is licensed under the MIT License.