Tags: awslabs/amazon-kinesis-video-streams-pic
Tags
release candidate (#244) * Fix definition of LP64 for gcc * Use INT64 instead of LL so the type is will consistent across platforms * Modify threadpool teardown (#218) * Modify threadpool teardown to avoid unlocking mutexes from a different thread than locked it. * Remove an extra comma that made the format incorrect (#216) * Use const PCHAR so C++ compilers won't complain about passing strings constants (#215) * Add stream state machine iterator (#220) * Define iterator function, add a Stream member var * Replace recursive stepState calls with iterator * Replace all Stream stepState calls with iterator * Fix existing comment typos * CLANG * Rename 'keepIterating' member to be more clear * Add null pointer checks to iterator * CLANG * Stylistic change based on precedent * Remove redundant pStateMachine null check * Remove another redundant null pointer check * Fix null checks * Add timerqueuekick and checkstatemachinetransition (#221) * Add timerqueuekick and checkstatemachinetransition * Increase sleep at end of threadpool test to give spawned threads time to free their memory * Utils - Remove duplicate error code * m1 build * update cmake * fix incompatible pointer error * fix incompatible pointer error * run tests * cmake once * no typecasting * Revert "no typecasting" This reverts commit f2096b8. * Revert "Revert "no typecasting"" This reverts commit 7d53af0. * everything working * Add gcc-M1 CI test (#225) * Add gcc M1 test * testing * more * Update gcc path * Try using LLP64 for Apple * Add is null or empty * Update CommonDefs.h (#229) * Fix 32 bit crash issue (#230) * fix 32-bit crash in threadpool and possible seg fault if get item was null * clang format * remove null item and keep iterating * properly free threadpool if failed to create (#231) * address compiler warnings, cleanup ci (#232) * remove unnecessary double compiles, separate out test ci run into a separate section, address compile warning * address multiple compile warnings * address more compiler warnings for linux, separate out few more tests from build in ci * move stuff around * use consistent flags in pic library and test * clang format * only run test if flag FIXUP_ANNEX_B_TRAILING_NALU_ZERO is set, previously we re-built pic with this flag for the tests but the test itself segfaults if it's not set * put tests behind ifdefs which require flags set * tweak ci yml * more ifdefing tests * more ifdefs for tests * more changes * correct path for tsan/ubsan suppressions files * address warning for linux implicit decl of pthread_getname_np * define only once * more ifdefs * more ifdefs for tests * ifdef more tests * more ifdefs * more ifdefs * more ifdefs * more ifdefs * more ifdefs * if thread count is going to overflow, return 0 * another if/def * add build and testing with ALIGNED_MEMORY_MODEL and FIXUP_ANNEX_B_TRAILING_NALU_ZERO to test both paths in CI * more ifdefs for unalign access * more ifdefs for ubsan * ifdef more tests for aligned memory model * more memory aligned ifdefs for tests * more if defs in tests * make sure correct number of bytes for snprintf * State machine name for logs (#233) * Add state machine tag * Add tag for client state machine * Use char with fixed size * Use default tags * Random string * Allow tag setting for state machines * Add unit tests * Free state machine * Use createStateMachineWithTag instead * Rename * Address comments * Silent (#234) * Remove const in the api definition for getStateMachineName (#235) * char* * Remove const * Enable tsan (#236) * enable tsan * lock access to global count to address tsan violation * remove globals, use struct to thread * some changes * lock to protect data race in case of canceled thread still * lock around terminate in tests for tsan data race * free mutex in tests * Release build (#237) * Switch to ASSERT_EQ from assert to prevent release build optimization * Fix assert * Cleanup log statements * Threadpool tsan (#238) * Protect accessing threadpool with threaddata lock * Fix potential seg fault from attempting to access queue after threadpool has been destroyed by locking the dataMutex around that queue, and having the destructor pass waiting tasks onto the queue when threads are stuck waiting for on said queue * Fix typo from merge * Reduce sleeps in threadpool * Lock test utils mutex, and check for termination in actors before assigning pQueue * need allocators set check * wait on teardown of test threads * removing accessing gTerminateCount in the for loop * longer sleep to wait for threads to free their heap * dramatically increase sleep * Add sleep in the heap check in case it's still waiting for a thread to exit * Infinite loop if memory isn't clear. Tests will timeout on GHA if the memory leak actually exists. If it's just waiting for threads to clean up, then it will keep waiting. * Remove sleep from termination task * Assert handle (#239) * Use abort instead * custom assert * Fix definition * Fix fprintf * Simple assert * Clang formatted * Fix build issue * safe gmtime, guard with mutex since not thread safe on non windows pl… (#240) * safe gmtime, guard with mutex since not thread safe on non windows platforms * clang format * add cygwin to keep original gmtime since pthread isn't used there in our thread impl * address pr comments, fix compile warnings in threadpool * missed the extern variable name change * add EXPECT_EQ around thread create and thread join calls in the unit tests, possibly address tsan complain where it it looks like multiple tests executing at the same time * unlock stream mutex before acquiring the fom mutex to avoid deadlock with putframe (#242) * Update ci.yml using xcode 14.3.1 results in a linker issue with gcc, 15.1 fixes the issue, trying 15.2 since this appears to be the latest available. * Update ci.yml ci.yml formatting * Reset current index in case it overflows out of range (#245) * Add PR description lint (#248) * Merge pull request #249 from awslabs/fix-read-windows Fix readFile on Windows --------- Co-authored-by: Dave Johansen <davejohansen@gmail.com> Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Stefan Kieszkowski <85728496+stefankiesz@users.noreply.github.com> Co-authored-by: Jeremy Gunawan <sirknightj@gmail.com> Co-authored-by: Niyati Maheshwari <niyatim@amazon.com> Co-authored-by: Niyati Maheshwari <niyatim23@gmail.com> Co-authored-by: Jeremy Gunawan <jggunawa@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu>
Release 1 0 0 (#207) * Added README instruction to use devleop, and added develop to travis.yml (#135) * Adding a generic utility for exponential backoff retries (#136) * Adding a utility for exponential backoff retries * Adding a utility to compute power Testing: Added unit tests for all relevant changes * updated cmake verison from 2.8 to 3.6.3 (#139) * KVS Events feature (images/notifications) (#138) * Images feature. Tests still incomplete, comments to follow. * Address code review comments, fixed UT, formatted. * Fixed compile errors from enum renames and struct type changes * Unit test floating point issue from %i being able to divide by 0. * Removed variable declaration from for loop and simplified redundant if statements. * Changed enum to be bit shifted ints, allowed to bitwise or multiple events into a single API call * updated struct to include versioning * Style and comment update * removed outdated comment * Define moved back into an enum, macros added for internal bit-wise operations with associated enum * added error messages to default case * spacing for strings tag was incorrect * Additional MKV api tests to ensure the correct formatting of MKV event fragments * Missing declaration in Stream.h for putEventMetadata (#142) * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * Log git hash * Adding error handler in PIC state machine and updating client/stream state machines to use the error handler on state transitions (#143) * The commit includes following - (1) Refactoring Exponential backoff utility by changing name(s) to use 'RetryStrategy' (2) Minor logging improvements in Exponential backoff utility (3) Adding KVS retry strategy and related callbacks to be used in client and streams (4) Adding a new error handler in PIC state machine (5) Updating KVS client and stream state machines to use PIC error handler and exponential backoff retry strategy on encountering error while state transitions * - Adding client callback to get retry strategy. This is particularly needed for unit testing - Addressing PR feedback * - Updating UTs to handle memory leak for offline streaming - Addressing PR feedback * Addressing PR feedback * Update Client.c trigger travis CI * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * Log git hash Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Expose retry count * Move to callback model to get count and move retry state out of public header * Fix potential NULL dereference * Address comments * Move state back to public header since unit tests are failing. Tests will have to fixedi n another PR * Incorporating full jitter implementation of exponential backoff algorithm * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries (#155) * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries * Remove check for strategy type (#156) * Test exp for same state * Remove check for strategy type until we handle the STRATEGY_TYPE placement appropriately * Decoupling retry strategy callbacks from the actual retry strategy, adding retry strategy type check before casting retry strategy state * Fix PR feedback and add test for getExponentialBackoffRetryCount * Add missing copy of retry strategy callbacks while creating kinesis video client and fix broken unit test * Crate retry strategy after fixing up device info (#158) * more specific service call results, will be used for clock skew auto correction (#159) * Fix up bad log message (#160) * setup ga; disable travis on develop * fix clang-format check on ga * fix ci file name * Add retry coumt metric * Fix %lu in logging * Trigger CI * Fix format specifiers * fix wiindows, clang, ubsan * fix windows, clang * fix windows, clang * fix clang-format * fix windows * fix windows * fix windows * fix windows * fix windows * fix ubsan and windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build with powershell * fix windows build with powershell * fix windows build with powershell * update call path windows * update call path windows * use aws-actions for creds * add region * fix aws-actions * fix aws-actions * fix aws-actions * use environment * unset session token * unset session token * unset session token * unsetting the token * fix builds * block tags before first cluster, post tags after (#151) * block tags before first cluster, post tags after * missing bool flip * correction to size expectations * use clang 7 * remove continue-on-error * setup arm cross-compilation with qemu * fix msan and ubsan * fix windows build * fix windows build; remove export * fix test instruction * fix test instruction * fix test instruction; get pwd and dir * fix test instruction with backslash * find current directory * check the contents of the build folder * execute the test * disable windows defender * disable windows defender * disable windows defender * fix typo * use docker image for windows * remove msvc action * remove msvc action * use msvc 2017 * use windows 2016 * use vcvars64 * run tests * check verbose logs * update travis and ga for working builds * fix windows build on travis * fix mac builds on ga * update cmake * fix mac builds * fix windows build * initialize firstFrameFirst * fix windows build * fix working builds * move windows build from travis to ga * fix indentation * run tests with arm * fix arm * remove exports, use latest, fix firstFramefirst * set log-level to 2, update mac build name, remove windows from travis * check CC * remove echo * update windows log level * update cross-compilation builds for ga * There are several missing 'accepted' states from the Stream State mac… (#148) * There are several missing 'accepted' states from the Stream State machine * Clang format * Extra accepted states found in Client state machine * I still forget to clang format... * Update StreamState.c trigger travis ci * needed for tagResourceResult Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * remove travis.yml * Give Frame a name * Develop (#177) * fix deadlock * fix semaphore release and lock release order * create new mutex around putFrame API call, necessary due to logStreamMetric insidue putframe needing to acquire streamListLock which would otherwise let other threads call putFrame unless we create a new high level lock * set client after we verify it is not NULL Co-authored-by: Hassan Sahibzada <hassan.sahibzada@datadoghq.com> * Rebase develop with master (#193) * Testing images stuff (#172) * debug messages around ordering * Changing start check * debug message * changing debug statement condition * Block event metadata when the mkvgenerator has not moved past the header yet * more logs * != -> == * Duplicate event tracking * Duplicate test added, tests that no longer apply have been removed * Clang format * fixing clang format test, removing unnecessary comment * Undefined values of frame struct * Merge develop into master (#178) * Added README instruction to use devleop, and added develop to travis.yml (#135) * Adding a generic utility for exponential backoff retries (#136) * Adding a utility for exponential backoff retries * Adding a utility to compute power Testing: Added unit tests for all relevant changes * updated cmake verison from 2.8 to 3.6.3 (#139) * KVS Events feature (images/notifications) (#138) * Images feature. Tests still incomplete, comments to follow. * Address code review comments, fixed UT, formatted. * Fixed compile errors from enum renames and struct type changes * Unit test floating point issue from %i being able to divide by 0. * Removed variable declaration from for loop and simplified redundant if statements. * Changed enum to be bit shifted ints, allowed to bitwise or multiple events into a single API call * updated struct to include versioning * Style and comment update * removed outdated comment * Define moved back into an enum, macros added for internal bit-wise operations with associated enum * added error messages to default case * spacing for strings tag was incorrect * Additional MKV api tests to ensure the correct formatting of MKV event fragments * Missing declaration in Stream.h for putEventMetadata (#142) * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * Log git hash * Adding error handler in PIC state machine and updating client/stream state machines to use the error handler on state transitions (#143) * The commit includes following - (1) Refactoring Exponential backoff utility by changing name(s) to use 'RetryStrategy' (2) Minor logging improvements in Exponential backoff utility (3) Adding KVS retry strategy and related callbacks to be used in client and streams (4) Adding a new error handler in PIC state machine (5) Updating KVS client and stream state machines to use PIC error handler and exponential backoff retry strategy on encountering error while state transitions * - Adding client callback to get retry strategy. This is particularly needed for unit testing - Addressing PR feedback * - Updating UTs to handle memory leak for offline streaming - Addressing PR feedback * Addressing PR feedback * Update Client.c trigger travis CI * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * Log git hash Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Expose retry count * Move to callback model to get count and move retry state out of public header * Fix potential NULL dereference * Address comments * Move state back to public header since unit tests are failing. Tests will have to fixedi n another PR * Incorporating full jitter implementation of exponential backoff algorithm * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries (#155) * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries * Remove check for strategy type (#156) * Test exp for same state * Remove check for strategy type until we handle the STRATEGY_TYPE placement appropriately * Decoupling retry strategy callbacks from the actual retry strategy, adding retry strategy type check before casting retry strategy state * Fix PR feedback and add test for getExponentialBackoffRetryCount * Add missing copy of retry strategy callbacks while creating kinesis video client and fix broken unit test * Crate retry strategy after fixing up device info (#158) * more specific service call results, will be used for clock skew auto correction (#159) * Fix up bad log message (#160) * setup ga; disable travis on develop * fix clang-format check on ga * fix ci file name * Add retry coumt metric * Fix %lu in logging * Trigger CI * Fix format specifiers * fix wiindows, clang, ubsan * fix windows, clang * fix windows, clang * fix clang-format * fix windows * fix windows * fix windows * fix windows * fix windows * fix ubsan and windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build with powershell * fix windows build with powershell * fix windows build with powershell * update call path windows * update call path windows * use aws-actions for creds * add region * fix aws-actions * fix aws-actions * fix aws-actions * use environment * unset session token * unset session token * unset session token * unsetting the token * fix builds * block tags before first cluster, post tags after (#151) * block tags before first cluster, post tags after * missing bool flip * correction to size expectations * use clang 7 * remove continue-on-error * setup arm cross-compilation with qemu * fix msan and ubsan * fix windows build * fix windows build; remove export * fix test instruction * fix test instruction * fix test instruction; get pwd and dir * fix test instruction with backslash * find current directory * check the contents of the build folder * execute the test * disable windows defender * disable windows defender * disable windows defender * fix typo * use docker image for windows * remove msvc action * remove msvc action * use msvc 2017 * use windows 2016 * use vcvars64 * run tests * check verbose logs * update travis and ga for working builds * fix windows build on travis * fix mac builds on ga * update cmake * fix mac builds * fix windows build * initialize firstFrameFirst * fix windows build * fix working builds * move windows build from travis to ga * fix indentation * run tests with arm * fix arm * remove exports, use latest, fix firstFramefirst * set log-level to 2, update mac build name, remove windows from travis * check CC * remove echo * update windows log level * update cross-compilation builds for ga * There are several missing 'accepted' states from the Stream State mac… (#148) * There are several missing 'accepted' states from the Stream State machine * Clang format * Extra accepted states found in Client state machine * I still forget to clang format... * Update StreamState.c trigger travis ci * needed for tagResourceResult Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * remove travis.yml * Give Frame a name * Develop (#177) * fix deadlock * fix semaphore release and lock release order * create new mutex around putFrame API call, necessary due to logStreamMetric insidue putframe needing to acquire streamListLock which would otherwise let other threads call putFrame unless we create a new high level lock * set client after we verify it is not NULL Co-authored-by: Hassan Sahibzada <hassan.sahibzada@datadoghq.com> * fix test Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Anurag Katey <kateyanurag@users.noreply.github.com> Co-authored-by: Niyati Maheshwari <niyatim23@gmail.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> Co-authored-by: Anurag Katey <akatey@amazon.com> Co-authored-by: Dave Johansen <dave@pitchaware.com> Co-authored-by: Hassan Sahibzada <hassan.sahibzada@datadoghq.com> * Fail create call with flag 1 4 1 (#181) * Modify jump from describe to create call based on a flag * Fix up stream info * fix printing, add unit test * update gtest to newer version due to gcc > 11.2 causing failures, fixed in gtest 1.12.1, replaced deprecated macros * clang-7 not available on newer Ubuntu/CI hosts, modifying to clang which should always work as its a link to an availble version not a specific version that needs constant updating * missed explicit reference to clang-7, replace with general * address comment about naming Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Fix CI * Test producer C build with removing justStartedStreaming flag * Add completion timeout configurability * Fix bug in unit test * [ENHANCEMENT] Use new log level (#187) * Add new log level * Modify file logger * Clean up logger functionality * Updated the images * Fix windows issue * Fix windows issue * Fix windows issue --------- Co-authored-by: Divya Sampath Kumar <divku@amazon.com> --------- Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: Anurag Katey <kateyanurag@users.noreply.github.com> Co-authored-by: Niyati Maheshwari <niyatim23@gmail.com> Co-authored-by: Anurag Katey <akatey@amazon.com> Co-authored-by: Dave Johansen <dave@pitchaware.com> Co-authored-by: Hassan Sahibzada <hassan.sahibzada@datadoghq.com> Co-authored-by: Divya Sampath Kumar <divku@amazon.com> * Fix ubsan failure (#197) * Fix ubsan failure * Bring back 4.4 * Add threadpool (#199) * Thread safe blocking queue implemented * full suite of unit tests for threadblockingqueue, and added tests and functionality for 'empty' semaphore * remove checks in destructor * Initial implementation commit of threadpool * * Fixed bugs in Semaphore.c *lossy signal *race condition deadlock in free *ununitialized atomics * Added corresponding tests to complete missing coverage * Added tests for Threadpool * Removed unused tests * clang formatting * Address sanitizer fixes * addressing more address sanitizer issues * Accidental deletion undone * Fixing windows build since it doesn't support INT32_MAX macro * Allow threadgetcount test some time for threads to settle to get expected counts * Fix windows build issues * For loop variable initialized outside for loop for some compilers * Added comments to header file, changed valid inputs for threadpoolCreate to require min & max to be > 0 * Fixup tests to account for new min/max > 0 requirement --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Remove INLINE from functions that are declared but not defined because it's not actually doing anything and creating warnings (#201) * Test logger (#202) * Add string length checks for events (#203) * Clang format (#204) * Remove printing cpd * Parellel build only the failing test * Release 1.0.0 * Log the content type when they don't match (#200) --------- Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Anurag Katey <kateyanurag@users.noreply.github.com> Co-authored-by: Niyati Maheshwari <niyatim23@gmail.com> Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: Anurag Katey <akatey@amazon.com> Co-authored-by: Dave Johansen <dave@pitchaware.com> Co-authored-by: Hassan Sahibzada <hassan.sahibzada@datadoghq.com> Co-authored-by: Divya Sampath Kumar <divku@amazon.com> Co-authored-by: Dave Johansen <davejohansen@gmail.com>