|
CWB
|
#include <math.h>#include "../cl/cl.h"#include "../cl/globals.h"#include "../cl/corpus.h"#include "../cl/attributes.h"#include "../cl/storage.h"#include "../cl/bitio.h"#include "../cl/compression.h"| #define log2 | ( | x | ) | (log(x)/log(2.0)) |
| void compress_reversed_index | ( | Attribute * | attr, |
| char * | output_fn | ||
| ) |
Compresses the reversed index of a p-attribute.
| attr | The attribute to compress the index of. |
| output_fn | Base name for the compressed RDX files to be written (if this is null, filenames will be taken from the attribute). |
References _Attribute::any, BFclose(), BFflush(), BFopen(), BFposition(), CDA_OK, cl_delete_stream(), cl_errno, cl_error(), cl_id2freq(), cl_max_cpos(), CL_MAX_FILENAME_LENGTH, cl_max_id(), cl_new_stream(), cl_read_stream(), codepos, CompCompRF, CompCompRFX, component_full_name(), compressrdx_cleanup(), CompRevCorpus, CompRevCorpusIdx, compute_ba(), corpus_id, debug, debug_output, ensure_component(), NwriteInt(), and write_golomb_code().
Referenced by main().
| void compressrdx_cleanup | ( | int | error_code | ) |
Cleans up memory prior to an error-prompted exit.
| error_code | Value to be returned by the program when it exits. |
References cl_delete_corpus(), and debug_output.
Referenced by compress_reversed_index(), compressrdx_usage(), decompress_check_reversed_index(), and main().
| void compressrdx_usage | ( | char * | msg, |
| int | error_code | ||
| ) |
Prints a usage message and exits the program.
| msg | A message about the error. |
| error_code | Value to be returned by the program when it exits. |
References compressrdx_cleanup(), progname, and VERSION.
Referenced by main().
| void decompress_check_reversed_index | ( | Attribute * | attr, |
| char * | output_fn | ||
| ) |
Checks a compressed reversed index for errors by decompressing it.
This function this assumes that compress_reversed_index() has been called beforehand and made sure that the _uncompressed_ index is used by CL access functions.
| attr | The attribute to check the index of. |
| output_fn | Base name for the compressed RDX files to be read (if this is null, filename swill be taken from the attribute). |
References _Attribute::any, BFclose(), BFflush(), BFopen(), CDA_OK, cl_delete_stream(), cl_errno, cl_error(), cl_id2freq(), cl_max_cpos(), CL_MAX_FILENAME_LENGTH, cl_max_id(), cl_new_stream(), cl_read_stream(), CompCompRF, CompCompRFX, component_full_name(), compressrdx_cleanup(), CompRevCorpus, CompRevCorpusIdx, compute_ba(), corpus_id, debug, debug_output, and read_golomb_code_bf().
Referenced by main().
| int main | ( | int | argc, |
| char ** | argv | ||
| ) |
Main function for cwb-compress-rdx.
| argc | Number of command-line arguments. |
| argv | Command-line arguments. |
References _Attribute::any, ATT_POS, TCorpus::attributes, cl_new_attribute_oldstyle(), cl_new_corpus(), cl_standard_registry(), compress_reversed_index(), compressrdx_cleanup(), compressrdx_usage(), corpus_id, debug, debug_output, decompress_check_reversed_index(), DEFAULT_ATT_NAME, progname, and registry_directory.
| int codepos = 0 |
stores current position in a bit-write-file
Referenced by compress_reversed_index().
Record for the corpus we are working on.
Referenced by cl_cpos2id(), cl_delete_attribute(), cl_max_cpos(), cl_new_corpus(), cqi_find_corpus(), do_cqi_cqp_list_subcorpora(), eval_constraint(), load_corpusnames(), main(), read_mapping(), and setup_attribute().
| char* corpus_id = NULL |
CWB id of the corpus we are working on.
Referenced by compress_reversed_index(), compute_code_lengths(), decode_add_attribute(), decode_check_huff(), decode_verify_print_value_list(), decompress_check_reversed_index(), lexdecode_show(), and main().
| int debug = 0 |
debug level
Referenced by compress_reversed_index(), decompress_check_reversed_index(), encode_generate_registry_file(), encode_parse_options(), main(), range_declare(), and sencode_parse_options().
| FILE* debug_output |
where debug messages are to be sent to (stderr)
Referenced by compress_reversed_index(), compressrdx_cleanup(), decompress_check_reversed_index(), do_MUQuery(), do_StandardQuery(), do_TABQuery(), and main().
| char* progname = NULL |
Name of the program.
1.7.3