TTS results are huge (#12)

Activities

wojciech.kapcia@tigase.net opened 7 years ago

Due Date
2018-04-26

We are growing number of test cases, and we are storing quite detailed log out put (which very often is helpful), but this results in having a complete log for one database for one version sum up to 60MB, and current history of test runs already occupies 11G. We should review debugging and logging options and:

review and logs (even with debug) to so there won't be duplicates and excessive strings without much value;
add "debug" option so by default only general output would be logged.

jenkins@build:/home/jenkins/nightlies/tts-ng-tests$ du -h -d 1 test-results/7.2.0-SNAPSHOT-b4912/all/
61M	test-results/7.2.0-SNAPSHOT-b4912/all/derby
60M	test-results/7.2.0-SNAPSHOT-b4912/all/sqlserver
59M	test-results/7.2.0-SNAPSHOT-b4912/all/mongodb
58M	test-results/7.2.0-SNAPSHOT-b4912/all/postgresql
237M	test-results/7.2.0-SNAPSHOT-b4912/all/

jenkins@build:/home/jenkins/nightlies/tts-ng-tests$ du -h -d 1 test-results/
175M	test-results/7.2.0-SNAPSHOT-b4909
228M	test-results/7.2.0-SNAPSHOT-b4879
227M	test-results/7.2.0-SNAPSHOT-b4877
221M	test-results/7.2.0-SNAPSHOT-b4870
189M	test-results/7.2.0-SNAPSHOT-b4989
205M	test-results/7.2.0-SNAPSHOT-b4866
244M	test-results/7.2.0-SNAPSHOT-b4916
182M	test-results/7.2.0-SNAPSHOT-b4961
236M	test-results/7.2.0-SNAPSHOT-b4937
230M	test-results/7.2.0-SNAPSHOT-b4894
188M	test-results/7.2.0-SNAPSHOT-b4959
165M	test-results/7.2.0-SNAPSHOT-b4846
237M	test-results/7.2.0-SNAPSHOT-b4899
188M	test-results/7.2.0-SNAPSHOT-b4945
188M	test-results/7.2.0-SNAPSHOT-b4948
228M	test-results/7.2.0-SNAPSHOT-b4880
56K	test-results/7.2.0-SNAPSHOT-b4906
5.0M	test-results/7.2.0-SNAPSHOT-b4861
220M	test-results/7.2.0-SNAPSHOT-b4871
222M	test-results/7.2.0-SNAPSHOT-b4941
220M	test-results/7.2.0-SNAPSHOT-b4857
221M	test-results/7.2.0-SNAPSHOT-b4923
166M	test-results/7.2.0-SNAPSHOT-b4848
231M	test-results/7.2.0-SNAPSHOT-b4895
234M	test-results/7.2.0-SNAPSHOT-b4875
206M	test-results/7.2.0-SNAPSHOT-b4874
184M	test-results/7.2.0-SNAPSHOT-b4990
44K	test-results/7.2.0-SNAPSHOT-b4908
208M	test-results/7.2.0-SNAPSHOT-b4873
217M	test-results/7.2.0-SNAPSHOT-b4841
185M	test-results/7.2.0-SNAPSHOT-b4947
244M	test-results/7.2.0-SNAPSHOT-b4920
223M	test-results/7.2.0-SNAPSHOT-b4835
187M	test-results/7.2.0-SNAPSHOT-b4972
228M	test-results/7.2.0-SNAPSHOT-b4881
159M	test-results/7.2.0-SNAPSHOT-b4834
247M	test-results/7.2.0-SNAPSHOT-b4942
190M	test-results/7.2.0-SNAPSHOT-b4960
214M	test-results/7.2.0-SNAPSHOT-b4851
184M	test-results/7.2.0-SNAPSHOT-b4949
233M	test-results/7.2.0-SNAPSHOT-b4843
240M	test-results/7.2.0-SNAPSHOT-b4904
183M	test-results/7.2.0-SNAPSHOT-b4974
180M	test-results/7.2.0-SNAPSHOT-b4922
235M	test-results/7.2.0-SNAPSHOT-b4911
184M	test-results/7.2.0-SNAPSHOT-b4970
208M	test-results/7.2.0-SNAPSHOT-b4868
220M	test-results/7.2.0-SNAPSHOT-b4935
228M	test-results/7.2.0-SNAPSHOT-b4838
111M	test-results/7.2.0-SNAPSHOT-b4849
230M	test-results/7.2.0-SNAPSHOT-b4842
185M	test-results/7.2.0-SNAPSHOT-b4968
221M	test-results/7.2.0-SNAPSHOT-b4856
237M	test-results/7.2.0-SNAPSHOT-b4913
167M	test-results/7.2.0-SNAPSHOT-b4850
237M	test-results/7.2.0-SNAPSHOT-b4912
11G	test-results/

wojciech.kapcia@tigase.net commented 6 years ago

I decided to strip from the testng-results.xml all non-essential information (we only need run data to generate index page with pass/fail/times) which reduces ~25MB files to couple of KB (information is still present in generated HTML). This slashed space usage from 32G to 12G and also sped up subsequent generations as there is less XML data to parse.

Type	Task
Priority	Normal
Assignee	wojciech.kapcia@tigase.net
RedmineID	6067

Issue Votes (0)

Watchers (0)

Reference

tigase/tigase-tts-ng#12