Tigase Forge
Dashboards
Projects
Pull Requests
Issues
Builds
Packages
Code Search
tigase-http-api
Code
Pull Requests
Issues
List
Boards
Iterations
Builds
Statistics
Child Projects
OneDev 11.5.2
Documentation Support & Bug Report RESTful API Incompatibilities License Agreement Check Update
Projects tigase _server tigase-http-api Issues #111
DefaultLogic.removeExpired removal of slot failed (#111)
wojciech.kapcia@tigase.net opened 4 years ago 
[2021-03-31 17:37:42:739] [FINE    ] [  ConnectionOpenThread ] DefaultLogic.removeExpired()     : removal of slot failed
java.io.IOException: Could not remove file 689c2aca-9fb5-4884-805a-2801a667b7d2 from S3
	at tigase.extras.http.upload.S3Store.remove(S3Store.java:140)
	at tigase.http.upload.logic.DefaultLogic.removeExpired(DefaultLogic.java:161)
	at tigase.http.upload.ExpirationTask.run(ExpirationTask.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: WZ0KP1NV2CMA4S30; S3 Extended Request ID: nGMd09CP/6HQon+9Zre3p5ORVSqWN85XS4pwNGia4Z9J54mStNSisCBGnLcidwLpR08CpBX7EFs=; Proxy: null), S3 Extended Request ID: nGMd09CP/6HQon+9Zre3p5ORVSqWN85XS4pwNGia4Z9J54mStNSisCBGnLcidwLpR08CpBX7EFs=
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5155)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5101)
	at com.amazonaws.services.s3.AmazonS3Client.deleteObjects(AmazonS3Client.java:2279)
	at tigase.extras.http.upload.S3Store.remove(S3Store.java:138)
	... 8 more
Andrzej Wójcik (Tigase) commented 4 years ago

Might be caused by files with incorrect filenames being uploaded a while ago. It would be good to increase logging level and log list of keys which we wanted to delete to verify that.
Andrzej Wójcik (Tigase) commented 4 years ago

I've checked S3 and it looks like there is no folder named 689c2aca-9fb5-4884-805a-2801a667b7d2 while there should be if the file was uploaded. Most likely file failed to upload by we still have old entries in slots repository which is used as a source of the list of files to remove.
Andrzej Wójcik (Tigase) commented 4 years ago

It looks like the issue was caused by "deleting" an empty list of files with the use of the DeleteObjects AWS S3 action which expects a list of files. The exception reported is about XML not well-formed, so most likely not matching schema (object to remove tag may be required to appear at least one time).

I've applied a fix to our code to skip requests to AWS if there are no files in the slot (were not uploaded or were already removed).
issue 1 of 1
Type
Bug
Priority
Normal
Assignee
wojciech.kapcia@tigase.net
Version
tigase-server-8.2.0
Issue Votes (0)
Login to vote
Watchers (0)
Reference
tigase/_server/tigase-http-api#111
Please wait...
Page is in error, reload to recover