Text analysis and exact excerpts take different paths

Text analysis and exact excerpts take different paths A data-flow diagram generated by Archify. 01 / Saved inputs 02 / Text processing 03 / Saved text 04 / Interpretation 05 / Reader output Post text · Saved XML paragraphs · 01 / Saved inputs Post text Saved XML paragraphs Audited metadata · Status, credits, dates · 01 / Saved inputs Audited metadata Status, credits, dates NLTK cleaning · Tokenize + clean · 02 / Text processing NLTK cleaning Tokenize + clean Literal spans · Keep source wording · 02 / Text processing Literal spans Keep source wording Cleaned corpus · 299 included posts · 03 / Saved text Cleaned corpus 299 included posts Excerpt pool · 872 sentences · 03 / Saved text Excerpt pool 872 sentences MALLET fit · Eight-topic primary · 04 / Interpretation MALLET fit Eight-topic primary Select voices · 30 excerpts / 25 posts · 04 / Interpretation Select voices 30 excerpts / 25 posts Topic weights · Reused in charts · 05 / Reader output Topic weights Reused in charts Excerpt reader · Selected or broader · 05 / Reader output Excerpt reader Selected or broader saved text metadata cleaned forms model words saved weights literal text eligible posts filter spans reviewed IDs selected voices broader option Legend data store data flow