[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/logging lookup #1436

Open
wants to merge 123 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
b38c06e
Initial logging with request id
RiddhiBagadiaa Apr 15, 2024
57d0921
Introduce new logger struct
RiddhiBagadiaa Apr 16, 2024
753d957
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
b4644c7
create provider
RiddhiBagadiaa Apr 16, 2024
e8d694f
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
57c1ec1
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
995e0d3
logger for CreateProvider
RiddhiBagadiaa Apr 17, 2024
7ec0f9e
modifications
RiddhiBagadiaa Apr 17, 2024
40a341a
Upgrades rest of aws-sdk-go-v2 (#1423)
simba-git Apr 16, 2024
1cb4838
client side for createprovider
RiddhiBagadiaa Apr 17, 2024
abdc712
Update other create functions with new logger
RiddhiBagadiaa Apr 17, 2024
5b9448b
request id in context checks
RiddhiBagadiaa Apr 18, 2024
5d485ce
add logging to client and log message to list and get methods in main.go
RiddhiBagadiaa Apr 18, 2024
d78642d
use functions withresource and withprovider, more formatting changes
RiddhiBagadiaa Apr 19, 2024
86707f7
Feature/fast materialize/simba (#1419)
simba-git Apr 18, 2024
be2816e
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
6242179
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
ba5eab2
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
fa8fb93
modify withresource, debug withrequestid, add request to logger struct
RiddhiBagadiaa Apr 19, 2024
e98a793
trying to add and update resources
RiddhiBagadiaa Apr 19, 2024
3f64b76
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
e763329
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
cab9a06
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
3956a14
Merge branch 'main' into feature/logging-messages
RiddhiBagadiaa Apr 19, 2024
b9a11bb
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
5b29ed3
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
7ec39ed
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
5a2dcff
behave tests and updatecontext function
RiddhiBagadiaa Apr 23, 2024
641fc00
More logging functions work
RiddhiBagadiaa Apr 23, 2024
650efaa
create functions working
RiddhiBagadiaa Apr 23, 2024
2cb0777
unit tests and debugging
RiddhiBagadiaa Apr 24, 2024
3941e01
list and get functions in api/main.go
RiddhiBagadiaa Apr 24, 2024
d930d13
Update main.go
RiddhiBagadiaa Apr 24, 2024
5bb88bc
Update main.go
RiddhiBagadiaa Apr 25, 2024
8114c8b
Add Java as a dependency for client tests (#1431)
simba-git Apr 24, 2024
3483c94
Bugfix: Using auto variants as part of SQL transformations (#1430)
ahmadnazeri Apr 24, 2024
49906cb
Tests: Adding additional behave tests (#1424)
ahmadnazeri Apr 24, 2024
46b49f0
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
383d862
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
ec1d9c5
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
1dd1260
Merge branch 'main' into feature/logging-messages
RiddhiBagadiaa Apr 25, 2024
91a49e3
create functions in metadata done
RiddhiBagadiaa Apr 25, 2024
bb9c638
Update metadata.go
RiddhiBagadiaa Apr 25, 2024
51fe07e
Update metadata.go with list and get logging
RiddhiBagadiaa Apr 25, 2024
137f2bf
more logging to metadata.go
RiddhiBagadiaa Apr 25, 2024
798500b
Update metadata.go
RiddhiBagadiaa Apr 25, 2024
8625648
Add context to lookup
RiddhiBagadiaa Apr 25, 2024
a0a0e8a
add context to lookup
RiddhiBagadiaa Apr 25, 2024
a9925c0
Revert "Update metadata.go"
RiddhiBagadiaa Apr 26, 2024
6fb3bff
update propogate change in metadata.go
RiddhiBagadiaa Apr 26, 2024
ee280ef
update get and list functions in metadata.go
RiddhiBagadiaa Apr 26, 2024
17d1902
List functions and getModels in api/main.go
RiddhiBagadiaa Apr 26, 2024
dd546b8
Get functions in main.go
RiddhiBagadiaa Apr 26, 2024
92be596
remove extra files
RiddhiBagadiaa Apr 26, 2024
b5e4f19
modify namevariant mentions in client.go
RiddhiBagadiaa Apr 26, 2024
8a27efd
modify list functions in client.go and fix errors
RiddhiBagadiaa Apr 29, 2024
9accf5f
Update main.go
RiddhiBagadiaa Apr 29, 2024
9501d3c
Add resource info to the logger
RiddhiBagadiaa Apr 29, 2024
5973c07
fixing get equivalent
RiddhiBagadiaa Apr 29, 2024
4c0ea63
change errorf to errorw
RiddhiBagadiaa Apr 29, 2024
0222c27
manage withProvider and withResource
RiddhiBagadiaa Apr 30, 2024
53f3803
Fix test names
RiddhiBagadiaa Apr 30, 2024
b544dc6
fix broken tests
RiddhiBagadiaa Apr 30, 2024
7d26ec9
improve formatting, add info logs
RiddhiBagadiaa Apr 30, 2024
7d2c653
Update main.go
RiddhiBagadiaa Apr 30, 2024
394ff21
Merge branch 'main' into feature/logging-messages
RiddhiBagadiaa Apr 30, 2024
444fbc9
cleaning logging.go
RiddhiBagadiaa Apr 30, 2024
87e0758
did i mess up the merge
RiddhiBagadiaa Apr 30, 2024
18b8598
Update Package Deps (#1435)
Apr 30, 2024
7c69fdd
Initial logging with request id
RiddhiBagadiaa Apr 15, 2024
b2f2324
sugaredlogging to logger struct
RiddhiBagadiaa Apr 16, 2024
8ce90fd
create provider
RiddhiBagadiaa Apr 16, 2024
4bfd8f8
switch to custom logger to allow container to build
RiddhiBagadiaa Apr 16, 2024
4e4327c
Reformat logger statements without explicitly mentioned sugarerlogger
RiddhiBagadiaa Apr 16, 2024
822cc6e
Update other create functions with new logger
RiddhiBagadiaa Apr 17, 2024
f8f5c8f
request id in context checks
RiddhiBagadiaa Apr 18, 2024
f442cdf
Feature/fast materialize/simba (#1419)
simba-git Apr 18, 2024
822244d
Merge branch 'main' into feature/logging-messages
RiddhiBagadiaa May 1, 2024
33251aa
fixing merge bugs
RiddhiBagadiaa May 1, 2024
8ef4841
debug for go tests
RiddhiBagadiaa May 2, 2024
3fdf4f3
metadata test bug fix
RiddhiBagadiaa May 2, 2024
3129f91
branched off logging-messages. Go tests pass, behave tests are failing
RiddhiBagadiaa May 2, 2024
76a46f0
check for errors in get functions before calling withresource
RiddhiBagadiaa May 2, 2024
f17ab6d
Update resources.py
RiddhiBagadiaa May 2, 2024
2b7dc4c
fixed errors, behave tests are good to go?
RiddhiBagadiaa May 3, 2024
4d3620d
Merge branch 'feature/logging-resource-variant' into feature/logging-…
RiddhiBagadiaa May 3, 2024
3523ec9
Update resources.py
RiddhiBagadiaa May 4, 2024
c7f68c1
Correct arguments for GetEquivalent function
RiddhiBagadiaa May 5, 2024
c99a012
Update main.go
RiddhiBagadiaa May 5, 2024
e00f1eb
Update main.go
RiddhiBagadiaa May 6, 2024
82fa831
debugging
RiddhiBagadiaa May 6, 2024
ef616ea
Add logs and fix type in client.go
RiddhiBagadiaa May 6, 2024
0bb42d4
update values map in logger
RiddhiBagadiaa May 7, 2024
e799ff1
Update metadata.go
RiddhiBagadiaa May 7, 2024
3d8f82e
formatting
RiddhiBagadiaa May 7, 2024
ac391d8
Merge branch 'feature/logging-resource-variant' into feature/logging-…
RiddhiBagadiaa May 7, 2024
709a8b7
make getequivalent more compact
RiddhiBagadiaa May 7, 2024
eb2b439
get rid of WithRequestID
RiddhiBagadiaa May 7, 2024
a03f8c6
modify function name
RiddhiBagadiaa May 7, 2024
b298b06
resolve github comments
RiddhiBagadiaa May 7, 2024
10c65cb
Delete docker_logs.txt
RiddhiBagadiaa May 7, 2024
bf754be
clean up
RiddhiBagadiaa May 7, 2024
bd5821c
Update main.go
RiddhiBagadiaa May 8, 2024
ff65cdc
make getequivalent more compact
RiddhiBagadiaa May 7, 2024
8dc792e
get rid of WithRequestID
RiddhiBagadiaa May 7, 2024
2e021d6
modify function name
RiddhiBagadiaa May 7, 2024
34daf9b
resolve github comments
RiddhiBagadiaa May 7, 2024
162fd03
Delete docker_logs.txt
RiddhiBagadiaa May 7, 2024
b21bf08
clean up
RiddhiBagadiaa May 7, 2024
56c0871
Update main.go
RiddhiBagadiaa May 8, 2024
3a6bd34
add context for lookup
RiddhiBagadiaa May 8, 2024
33d64ba
change resource type in logging to const
RiddhiBagadiaa May 8, 2024
4bedc89
resolve github comments
RiddhiBagadiaa May 9, 2024
0e1cd4f
Update main.go
RiddhiBagadiaa May 9, 2024
6c5964c
Merge branch 'feature/logging-resource-variant' into feature/logging-…
RiddhiBagadiaa May 9, 2024
d3bb982
Update logging_test.go
RiddhiBagadiaa May 9, 2024
54f6e06
resolved comments from PR
RiddhiBagadiaa May 9, 2024
9d593b5
error and format clean up
RiddhiBagadiaa May 9, 2024
34eec55
Merge branch 'feature/logging-resource-variant' into feature/logging-…
RiddhiBagadiaa May 9, 2024
8333cf4
Merge branch 'main' into feature/logging-lookup
RiddhiBagadiaa May 13, 2024
15c5910
Remove unnecessary contexts, resolve PR comments
RiddhiBagadiaa May 13, 2024
bdd335c
Merge branch 'main' into feature/logging-lookup
RiddhiBagadiaa May 15, 2024
7d58dcf
Update serving.go
RiddhiBagadiaa May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
resolve github comments
  • Loading branch information
RiddhiBagadiaa committed May 8, 2024
commit 34daf9b8d29774d5e81bfd7eff2958fc81727eff
13 changes: 7 additions & 6 deletions client/src/featureform/resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -2311,15 +2311,16 @@ def config(self):
def _get_and_set_equivalent_variant(
resource_variant_proto, variant_field, stub
) -> Optional[str]:
rv_proto = None
if feature_flag.is_enabled("FF_GET_EQUIVALENT_VARIANTS", True):
res_pb = pb.ResourceVariant(**{variant_field: getattr(resource_variant_proto, variant_field)})
res_pb = pb.ResourceVariant(
**{variant_field: getattr(resource_variant_proto, variant_field)}
)
equivalent = stub.GetEquivalent(
pb.ResourceVariantRequest(
resource_variant=res_pb,
request_id=resource_variant_proto.request_id,
)
pb.ResourceVariantRequest(
resource_variant=res_pb,
request_id=resource_variant_proto.request_id,
)
)
rv_proto = getattr(resource_variant_proto, variant_field)

# grpc call returns the default ResourceVariant proto when equivalent doesn't exist which explains the below check
Expand Down
2 changes: 1 addition & 1 deletion coordinator/coordinator.go
Original file line number Diff line number Diff line change
Expand Up @@ -410,7 +410,7 @@ func (c *Coordinator) verifyCompletionOfSources(sources []metadata.NameVariant)
totalReady += 1
}
if sourceVariant.Status() == metadata.FAILED {
c.Logger.Infow("Source variant failed", "sourceVariant", sourceVariant)
c.Logger.Errorw("Source variant failed", "sourceVariant", sourceVariant)
wrapped := fferr.NewResourceFailedError(sourceVariant.Name(), sourceVariant.Variant(), fferr.SOURCE_VARIANT, fmt.Errorf("required dataset is in a failed state"))
wrapped.AddDetail("resource_status", sourceVariant.Status().String())
return wrapped
Expand Down
2 changes: 1 addition & 1 deletion metadata/metadata.go
Original file line number Diff line number Diff line change
Expand Up @@ -1989,7 +1989,7 @@ type sendFn func(proto.Message) error
type initParentFn func(name, variant string) Resource

func (serv *MetadataServer) genericCreate(ctx context.Context, res Resource, init initParentFn) (*pb.Empty, error) {
logger := logging.GetLoggerFromContext(ctx).WithValues(map[string]interface{}{"resource-id": res.ID()})
logger := logging.GetLoggerFromContext(ctx).WithResource(res.ID().Type.String(), res.ID().Name, res.ID().Variant)
logger.Debugw("Creating Generic Resource: ", res.ID().Name, res.ID().Variant)

id := res.ID()
Expand Down