university/BA/scripts/.git
2 years agofinish master umbau origin/master
Vincent Dahmen [Sun, 1 Sep 2019 18:13:23 +0000 (20:13 +0200)]
finish

2 years agoscore: dont stop the music
Vincent Dahmen [Sun, 11 Aug 2019 09:11:25 +0000 (11:11 +0200)]
score: dont stop the music

2 years agoscore: set new source for translation
Vincent Dahmen [Sat, 10 Aug 2019 20:57:28 +0000 (22:57 +0200)]
score: set new source for translation

2 years agoscore: works now with multiple domains and separate test data
Vincent Dahmen [Sat, 10 Aug 2019 20:20:03 +0000 (22:20 +0200)]
score: works now with multiple domains and separate test data

2 years agoeval/plots: disabled two functions because the data dimensions changed. Will fix...
Vincent Dahmen [Sat, 10 Aug 2019 09:37:19 +0000 (11:37 +0200)]
eval/plots: disabled two functions because the data dimensions changed. Will fix later

2 years agoeval/plots: make thins a little bit more beatiful
Vincent Dahmen [Fri, 9 Aug 2019 21:16:58 +0000 (23:16 +0200)]
eval/plots: make thins a little bit more beatiful

2 years agoscore: enables multi scoring
Vincent Dahmen [Fri, 9 Aug 2019 20:02:11 +0000 (22:02 +0200)]
score: enables multi scoring

2 years agoplots: adds domains
Vincent Dahmen [Fri, 9 Aug 2019 19:27:54 +0000 (21:27 +0200)]
plots: adds domains

2 years agoscore: removes the right files...
Vincent Dahmen [Fri, 9 Aug 2019 18:37:54 +0000 (20:37 +0200)]
score: removes the right files...

2 years agoscore: clean reference before teting
Vincent Dahmen [Fri, 9 Aug 2019 18:31:53 +0000 (20:31 +0200)]
score: clean reference before teting

2 years agosocre: diable nlg again
Vincent Dahmen [Fri, 9 Aug 2019 18:22:33 +0000 (20:22 +0200)]
socre: diable nlg again

2 years agoscore: fixes the same issue as last time...
Vincent Dahmen [Fri, 9 Aug 2019 17:07:11 +0000 (19:07 +0200)]
score: fixes the same issue as last time...

2 years agoscore: adds multi eval again
Vincent Dahmen [Fri, 9 Aug 2019 17:05:15 +0000 (19:05 +0200)]
score: adds multi eval again

2 years agoeval: small cleanup
Vincent Dahmen [Fri, 9 Aug 2019 16:28:27 +0000 (18:28 +0200)]
eval: small cleanup

2 years agoeval/data: adds support for domains
Vincent Dahmen [Fri, 9 Aug 2019 16:27:37 +0000 (18:27 +0200)]
eval/data: adds support for domains

2 years agoscore: dobby cries
Vincent Dahmen [Fri, 9 Aug 2019 16:05:50 +0000 (18:05 +0200)]
score: dobby cries

2 years agoscore: fixes stupid typo (again)
Vincent Dahmen [Fri, 9 Aug 2019 15:30:08 +0000 (17:30 +0200)]
score: fixes stupid typo (again)

2 years agotrain/score: do the output correctly too
Vincent Dahmen [Fri, 9 Aug 2019 12:47:41 +0000 (14:47 +0200)]
train/score: do the output correctly too

2 years agoscore: stupid stupid mistake... dobby is not proud
Vincent Dahmen [Fri, 9 Aug 2019 12:33:06 +0000 (14:33 +0200)]
score: stupid stupid mistake... dobby is not proud

2 years agoscore: use doamin references now
Vincent Dahmen [Fri, 9 Aug 2019 12:25:26 +0000 (14:25 +0200)]
score: use doamin references now

2 years agoscore: adds option to skip the training
Vincent Dahmen [Fri, 9 Aug 2019 12:08:42 +0000 (14:08 +0200)]
score: adds option to skip the training

2 years agoadds doman testing
Vincent Dahmen [Fri, 9 Aug 2019 12:00:49 +0000 (14:00 +0200)]
adds doman testing

2 years agoAAAAARGh should sleep soon
Vincent Dahmen [Fri, 9 Aug 2019 00:07:53 +0000 (02:07 +0200)]
AAAAARGh should sleep soon

2 years agofixes fck typo
Vincent Dahmen [Fri, 9 Aug 2019 00:06:20 +0000 (02:06 +0200)]
fixes fck typo

2 years agoscore: removes subdir rule, so train works on prepapred servers #HACK
Vincent Dahmen [Thu, 8 Aug 2019 23:58:54 +0000 (01:58 +0200)]
score: removes subdir rule, so train works on prepapred servers #HACK

2 years agoscore: again stupid debugging
Vincent Dahmen [Thu, 8 Aug 2019 23:49:48 +0000 (01:49 +0200)]
score: again stupid debugging

2 years agotrain/score: add some tests and noise
Vincent Dahmen [Thu, 8 Aug 2019 23:37:02 +0000 (01:37 +0200)]
train/score: add some tests and noise

2 years agotrain/score: adds nlp_eval
Vincent Dahmen [Thu, 8 Aug 2019 23:22:59 +0000 (01:22 +0200)]
train/score: adds nlp_eval

2 years agotrain/install: add nlg-eval to be installed
Vincent Dahmen [Thu, 8 Aug 2019 23:04:15 +0000 (01:04 +0200)]
train/install: add nlg-eval to be installed

2 years agoeval/plots: adds weird metric view
Vincent Dahmen [Thu, 8 Aug 2019 08:13:28 +0000 (10:13 +0200)]
eval/plots: adds weird metric view

2 years agoeval/plot: adds legendfor corpus stats
Vincent Dahmen [Wed, 7 Aug 2019 14:49:21 +0000 (16:49 +0200)]
eval/plot: adds legendfor corpus stats

2 years agoeval/plots: adds langauge comparison
Vincent Dahmen [Wed, 7 Aug 2019 14:16:27 +0000 (16:16 +0200)]
eval/plots: adds langauge comparison

2 years agoeval/plots: adds side constraint comparison
Vincent Dahmen [Wed, 7 Aug 2019 11:27:13 +0000 (13:27 +0200)]
eval/plots: adds side constraint comparison

2 years agotrain/score: fixes path prefix issue
Vincent Dahmen [Tue, 6 Aug 2019 21:37:37 +0000 (23:37 +0200)]
train/score: fixes path prefix issue

2 years agoeval/plots: finalizes hyperparam (awaiting new data)
Vincent Dahmen [Tue, 6 Aug 2019 21:22:11 +0000 (23:22 +0200)]
eval/plots: finalizes hyperparam (awaiting new data)

2 years agoeval/plots: adds trainings curve and hyperparameter (draft)
Vincent Dahmen [Tue, 6 Aug 2019 20:04:09 +0000 (22:04 +0200)]
eval/plots: adds trainings curve and hyperparameter (draft)

2 years agoeval/plots: add corpus plot
Vincent Dahmen [Tue, 6 Aug 2019 10:48:05 +0000 (12:48 +0200)]
eval/plots: add corpus plot

2 years agoeval: add metric swtich for everything
Vincent Dahmen [Mon, 5 Aug 2019 14:50:30 +0000 (16:50 +0200)]
eval: add metric swtich for everything

2 years agoeval/stats: perfect score now works with alls core types
Vincent Dahmen [Mon, 5 Aug 2019 11:18:28 +0000 (13:18 +0200)]
eval/stats: perfect score now works with alls core types

2 years agoI forgot to save this one, I guess
Vincent Dahmen [Mon, 5 Aug 2019 10:00:09 +0000 (12:00 +0200)]
I forgot to save this one, I guess

2 years agoeval/parse: adds score parsinG
Vincent Dahmen [Mon, 5 Aug 2019 09:45:50 +0000 (11:45 +0200)]
eval/parse: adds score parsinG

2 years agoscore: bleu lowwercase with correct prefix
Vincent Dahmen [Mon, 5 Aug 2019 08:54:53 +0000 (10:54 +0200)]
score: bleu lowwercase with correct prefix

2 years agoscore: fixed another small typo.. I should not code so late)
Vincent Dahmen [Mon, 5 Aug 2019 06:48:22 +0000 (08:48 +0200)]
score: fixed another small typo.. I should not code so late)

2 years agotrain/score: I am tired and I make mistaeks... like this little one with the var...
Vincent Dahmen [Mon, 5 Aug 2019 00:42:58 +0000 (02:42 +0200)]
train/score: I am tired and I make mistaeks... like this little one with the var in the wrong place... welll

2 years agotrain/scores: fixes darn subshell redirecting
Vincent Dahmen [Mon, 5 Aug 2019 00:37:58 +0000 (02:37 +0200)]
train/scores: fixes darn subshell redirecting

2 years agotrain/score: write scores into files
Vincent Dahmen [Mon, 5 Aug 2019 00:34:30 +0000 (02:34 +0200)]
train/score: write scores into files

2 years agoinstall/rouge: fiexs broken links
Vincent Dahmen [Mon, 5 Aug 2019 00:24:24 +0000 (02:24 +0200)]
install/rouge: fiexs broken links

2 years agotrain/install: rouge tests updated
Vincent Dahmen [Mon, 5 Aug 2019 00:22:38 +0000 (02:22 +0200)]
train/install: rouge tests updated

2 years agotrain/util: typo in url
Vincent Dahmen [Mon, 5 Aug 2019 00:19:17 +0000 (02:19 +0200)]
train/util: typo in url

2 years agotrain/scoring: adds rouge with full install
Vincent Dahmen [Mon, 5 Aug 2019 00:15:08 +0000 (02:15 +0200)]
train/scoring: adds rouge with full install

2 years agotrain/score: add some stupid lost lines
Vincent Dahmen [Sun, 4 Aug 2019 07:25:22 +0000 (09:25 +0200)]
train/score: add some stupid lost lines

2 years agotrain/score: bunch of smaller fixes
Vincent Dahmen [Sun, 4 Aug 2019 00:27:43 +0000 (02:27 +0200)]
train/score: bunch of smaller fixes

2 years agotrain/score: fixes detection of existing translations
Vincent Dahmen [Sat, 3 Aug 2019 23:44:17 +0000 (01:44 +0200)]
train/score: fixes detection of existing translations

2 years agotrain/score: fixes some stuff with the bleu scripts
Vincent Dahmen [Sat, 3 Aug 2019 23:34:55 +0000 (01:34 +0200)]
train/score: fixes some stuff with the bleu scripts

2 years agotrain/score: fixes src files for testings
Vincent Dahmen [Sat, 3 Aug 2019 23:13:25 +0000 (01:13 +0200)]
train/score: fixes src files for testings

2 years agotrain/score: use local src for testing
Vincent Dahmen [Sat, 3 Aug 2019 22:48:15 +0000 (00:48 +0200)]
train/score: use local src for testing

2 years agotrain/score: limits test lines to 1k
Vincent Dahmen [Sat, 3 Aug 2019 19:57:51 +0000 (21:57 +0200)]
train/score: limits test lines to 1k

2 years agotrain:install: adds pyrouge to be installed into the virutalenv
Vincent Dahmen [Sat, 3 Aug 2019 14:06:12 +0000 (16:06 +0200)]
train:install: adds pyrouge to be installed into the virutalenv

2 years agotrain/score: removes rename since it is not part of util smh
Vincent Dahmen [Sat, 3 Aug 2019 13:59:30 +0000 (15:59 +0200)]
train/score: removes rename since it is not part of util smh

2 years agotrain/score: move stuff around
Vincent Dahmen [Sat, 3 Aug 2019 13:54:55 +0000 (15:54 +0200)]
train/score: move stuff around

2 years agotrain/score: adds additional output debug
Vincent Dahmen [Sat, 3 Aug 2019 13:19:01 +0000 (15:19 +0200)]
train/score: adds additional output debug

2 years agotrain/score: use of more advanced parallel stuff
Vincent Dahmen [Sat, 3 Aug 2019 13:06:04 +0000 (15:06 +0200)]
train/score: use of more advanced parallel stuff

2 years agotrain/scripts: fixes typo for date params
Vincent Dahmen [Wed, 31 Jul 2019 18:44:13 +0000 (20:44 +0200)]
train/scripts: fixes typo for date params

2 years agotrain/score: adds additional scores and runs in sync
Vincent Dahmen [Wed, 31 Jul 2019 18:18:25 +0000 (20:18 +0200)]
train/score: adds additional scores and runs in sync

2 years agotrain/bash-scripts: remove email support for debugging
Vincent Dahmen [Tue, 30 Jul 2019 10:57:01 +0000 (12:57 +0200)]
train/bash-scripts: remove email support for debugging

2 years agoeval/parse: changes rule structure
Vincent Dahmen [Fri, 26 Jul 2019 12:53:44 +0000 (14:53 +0200)]
eval/parse: changes rule structure

2 years agoeval/hyper: fixes some sources and extends report to include all train runs
Vincent Dahmen [Fri, 26 Jul 2019 10:05:24 +0000 (12:05 +0200)]
eval/hyper: fixes some sources and extends report to include all train runs

2 years agoeval/scripts: why can I commit in itime?
Vincent Dahmen [Thu, 25 Jul 2019 14:21:55 +0000 (16:21 +0200)]
eval/scripts: why can I commit in itime?

2 years agoeval/hyperparams: adds ability to generate trainingsconfig
Vincent Dahmen [Fri, 19 Jul 2019 10:05:10 +0000 (12:05 +0200)]
eval/hyperparams: adds ability to generate trainingsconfig

2 years agoeval/stats: use true value comparison
Vincent Dahmen [Fri, 19 Jul 2019 10:04:47 +0000 (12:04 +0200)]
eval/stats: use true value comparison

2 years agoeval/parse: use actual default values instead of None
Vincent Dahmen [Fri, 19 Jul 2019 10:04:18 +0000 (12:04 +0200)]
eval/parse: use actual default values instead of None

2 years agotrain/train.sh: adapts new way of working with configs
Vincent Dahmen [Fri, 19 Jul 2019 10:03:12 +0000 (12:03 +0200)]
train/train.sh: adapts new way of working with configs

2 years agoeval/parse: its too late and something just broke
Vincent Dahmen [Fri, 19 Jul 2019 03:02:29 +0000 (05:02 +0200)]
eval/parse: its too late and something just broke

2 years agoeval/hyperparams: pretty print everything (including sort by score)
Vincent Dahmen [Tue, 16 Jul 2019 15:34:06 +0000 (17:34 +0200)]
eval/hyperparams: pretty print everything (including sort by score)

2 years agoeval/hyperparams: full output including missing, invalid and table
Vincent Dahmen [Tue, 16 Jul 2019 15:33:44 +0000 (17:33 +0200)]
eval/hyperparams: full output including missing, invalid and table

2 years agoeval/hyperparams. moved everything a little bit and use only dicts now
Vincent Dahmen [Sat, 13 Jul 2019 11:59:39 +0000 (13:59 +0200)]
eval/hyperparams. moved everything a little bit and use only dicts now

2 years agoeval/hyperparams: switdches from tuples to dicts... nearly done!
Vincent Dahmen [Sat, 13 Jul 2019 03:00:45 +0000 (05:00 +0200)]
eval/hyperparams: switdches from tuples to dicts... nearly done!

2 years agoeval/hyperparams: fixes multiple issues and adds multi target runs
Vincent Dahmen [Fri, 12 Jul 2019 23:52:39 +0000 (01:52 +0200)]
eval/hyperparams: fixes multiple issues and adds multi target runs

2 years agotrain: fixers config path (hopoefully for the last time) and allows single runs to...
Vincent Dahmen [Wed, 10 Jul 2019 21:10:32 +0000 (23:10 +0200)]
train: fixers config path (hopoefully for the last time) and allows single runs to fail

2 years agotrain: I HATE THOSE SMALL THINGSgit add train/train.sh
Vincent Dahmen [Mon, 8 Jul 2019 22:48:49 +0000 (00:48 +0200)]
train: I HATE THOSE SMALL THINGSgit add train/train.sh

2 years agotrain: make sure queue gets deleted
Vincent Dahmen [Mon, 8 Jul 2019 19:41:08 +0000 (21:41 +0200)]
train: make sure queue gets deleted

2 years agotrain: time each run
Vincent Dahmen [Mon, 8 Jul 2019 15:01:35 +0000 (17:01 +0200)]
train: time each run

2 years agotrain/train: use queue instead simple files
Vincent Dahmen [Mon, 8 Jul 2019 10:47:57 +0000 (12:47 +0200)]
train/train: use queue instead simple files

2 years agoeval/mixed: prints missing stuff
Vincent Dahmen [Sat, 6 Jul 2019 14:07:00 +0000 (16:07 +0200)]
eval/mixed: prints missing stuff

2 years agoeval/fuckup: not enough concentration breakseverything... going to strip some stuff now
Vincent Dahmen [Sat, 6 Jul 2019 12:15:26 +0000 (14:15 +0200)]
eval/fuckup: not enough concentration breakseverything... going to strip some stuff now

2 years agoeval/scripts: post night dump
Vincent Dahmen [Sat, 6 Jul 2019 07:39:41 +0000 (09:39 +0200)]
eval/scripts: post night dump

2 years agoeval/parser: can now work on dirs and haz fancy bar
Vincent Dahmen [Sat, 6 Jul 2019 00:49:19 +0000 (02:49 +0200)]
eval/parser: can now work on dirs and haz fancy bar

2 years agoeval/parser: unkonwn side step
Vincent Dahmen [Fri, 5 Jul 2019 21:42:19 +0000 (23:42 +0200)]
eval/parser: unkonwn side step

2 years agoevaluation/parser: split everything into functions
Vincent Dahmen [Fri, 5 Jul 2019 19:58:29 +0000 (21:58 +0200)]
evaluation/parser: split everything into functions

2 years agoevaluation: adds first log parsing snippet
Vincent Dahmen [Fri, 5 Jul 2019 16:56:13 +0000 (18:56 +0200)]
evaluation: adds first log parsing snippet

2 years agotrain/score: adds ability to skip if existing translations have been found
Vincent Dahmen [Thu, 4 Jul 2019 15:29:07 +0000 (17:29 +0200)]
train/score: adds ability to skip if existing translations have been found

2 years agotrain/score: fix config path location
Vincent Dahmen [Wed, 3 Jul 2019 06:52:16 +0000 (08:52 +0200)]
train/score: fix config path location

2 years agoconfig/score: fixes path and reduce logfile size
Vincent Dahmen [Wed, 3 Jul 2019 06:40:06 +0000 (08:40 +0200)]
config/score: fixes path and reduce logfile size

2 years agotrain/score: simplifies multiple path
Vincent Dahmen [Wed, 3 Jul 2019 01:33:20 +0000 (03:33 +0200)]
train/score: simplifies multiple path

2 years agotrain7scroe: ship it hot
Vincent Dahmen [Wed, 3 Jul 2019 01:23:43 +0000 (03:23 +0200)]
train7scroe: ship it hot

2 years agotrain/score: fixes config path
Vincent Dahmen [Wed, 3 Jul 2019 01:21:54 +0000 (03:21 +0200)]
train/score: fixes config path

2 years agotrain/score: adds first script to calculate scoring for models using BLEU and default...
Vincent Dahmen [Wed, 3 Jul 2019 01:09:29 +0000 (03:09 +0200)]
train/score: adds first script to calculate scoring for models using BLEU and default validation data

2 years agotrain/mqtt: fix several smaller issues
Vincent Dahmen [Mon, 1 Jul 2019 19:26:01 +0000 (21:26 +0200)]
train/mqtt: fix several smaller issues

2 years agotrain/mqtt handler: fixes typo in if condition
Vincent Dahmen [Mon, 1 Jul 2019 11:28:48 +0000 (13:28 +0200)]
train/mqtt handler: fixes typo in if condition

2 years agoconfig/train: updates template with new epoch sizes
Vincent Dahmen [Fri, 28 Jun 2019 11:27:55 +0000 (13:27 +0200)]
config/train: updates template with new epoch sizes