WordIndex: WordIndex

  • Main Page
  • Modules
  • Namespaces
  • Classes
  • Files
  • Related Pages

WordIndex

0.0

WordIndex creates an alphabetically sorted list of words present in the input files and it reports the lines where those words occur. In addition it can also report the frequency of word occurrences in percent.

Besides this program, you may also be interested in GNU Style and Diction, in AntConc, a freeware concordance program for Windows, Macintosh OS X, and Linux, and in Concordance (commercial).

WordIndex is a makeover of one of my earlier C programs xref. The goal of writing xref was to become familiar with tokenizing input and programming an appropriate datastructure to store the words in. xref was written for BDSC 1.4 using WordStar on a Bondwell 12 transportable computer running CP/M 2.2.

Twenty years later, I was interested to see how I would write such a program in C++.

Notable aspects in which (the development of) this program differs from the historical one are

  • the use of the Standard Template Library [STL]
  • the treatment of the input files as collections [Wilson, 2007]
  • the use of a library to handle commandline options [TCLAP],
  • the use of IDEs [MS Visual Studio, Code::Blocks]
  • the use of a single (i.e. non-recursive) makefile [Miller, 1997]
  • the use of a refactoring tool [Visual Assist]
  • the use of a unit testing framework [Fructose]
  • the use of a tool to generate documentation from the program sources [Doxygen].
  • the use of a version control system [Subversion, TortoiseSVN].
  • assessing the software's complexity through software metrics tools [cccc, LocMetrix]

Contents

  • User Manual
  • On the Implementation
  • Software Metrics
  • Unit Tests
  • Literature References
  • Software References

WordIndex License

Copyright © 2007-2008, by Martin J. Moene <m.j.moene@eld.physics.LeidenUniv.nl>.

Permission to use, copy, modify, and distribute this software and its documentation under the terms of the GNU General Public License is hereby granted. No representations are made about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. See the GNU General Public License for more details.

Web page: http://www.eld.leidenuniv.nl/~moene/Home/projects/wordindex/

WordIndex Documentation © 2008 by Martin J. Moene


WordIndex: Module Index

  • Classes
  • Files
  • Related Pages

WordIndex Modules

Here is a list of all modules:
  • User Manual
  • On the Implementation
    • Reading Input Text
    • Storing Text and Linenumbers
  • Unit Tests
    • Logger Unit Test
    • Pair Unit Test
    • Tokenizer Unit Test
    • Utility Unit Test
    • WordIndex Unit Test
  • Software Metrics
    • Software Metrics Overview
    • Software Metrics Per Function


WordIndex: User Manual

User Manual

Here is WordIndex's help screen.
Usage: bin\wordindex.exe [option...] [file...]

  -h, --help          display this help and exit
  -a, --author        report authors name and e-mail [no]
      --version       report program and compiler versions [no]
  -v, --verbose       report ... [none]

  -f, --frequency     also report word frequency as d.dd% (n) [no]
  -l, --lowercase     transform words to lowercase [no]
  -r, --reverse       only collect keyword occurrences, see --keywords [no]
  -s, --summary       also report number of (key)words and references [no]

  -i, --input=file    read filenames from given file [standard input or given filenames]
  -o, --output=file   write output to given file [standard output]
  -k, --keywords=file read keywords to skip (stopwords) from given file [none]

Long options also may start with a plus, like: +help.

wordindex creates an alphabetically sorted index of words present in the
input files and it reports the lines where those words occur.
Words that are marked as keywords are excluded (see option --keywords).
Use option --reverse to only show the occurrences of keywords.

Words can be read from standard input, or from files specified on the command
line and from files that are specified in another file (see option --input).

A file that specifies input filenames may look as follows:
   # comment that extends to the end of the line ( ; also starts comment line)
   file1.txt file2.txt
   file3.txt

Example:
   echo hello world | wordindex +summary +frequency
       keywords  0
          words  2
     references  2

          hello  50% (1)  1
          world  50% (1)  1

Example:
   wordindex +lowercase file.txt | sort -n -k2 -r
This creates a list of lowercase words, sorted on frequency of occurrence.


WordIndex: On the Implementation

On the Implementation

Detailed Description

More description.

Modules

Reading Input Text
WordIndex uses the idea by Matthew Wilson [Wilson, 2007] to treat the input stream as a collection.
Storing Text and Linenumbers
WordIndex uses a word--linenumbers collection (a map) to store the words with the list of linenumbers where the word occurs.


WordIndex: Reading Input Text

Reading Input Text
[On the Implementation]

WordIndex uses the idea by Matthew Wilson [Wilson, 2007] to treat the input stream as a collection.

Here's an example how class Tokenizer treats an input stream as a collection.

// g++ -I../../src -o copy_tokens.exe copy_tokens.cpp

#include <iostream>
#include <iterator>
#include "Tokenizer.h"

using wordindex::Tokenizer;

namespace wordindex {

std::ostream& operator<< ( std::ostream& os, Tokenizer::value_type const value )
{
   return os << "(" << value.first << "," << value.second << ")";
}

}

void copy_tokens( std::istream& is, std::ostream& os )
{
   Tokenizer tokenizer( is );

   std::copy
   ( tokenizer.begin()              // input collection begin
   , tokenizer.end()                // input collection end
   , std::ostream_iterator<Tokenizer::value_type>( os, "\n" )
   );
}

int main()
{
   copy_tokens( std::cin, std::cout );
}

Running copy_tokens < copy_tokens.cpp gives the following output.

(g,1)
(i,1)
(src,1)
(o,1)
(copy_tokens,1)
(exe,1)
(copy_tokens,1)
(cpp,1)
(include,3)
(iostream,3)
(include,4)
(iterator,4)
(include,5)
(tokenizer,5)
(h,5)
(using,7)
(wordindex,7)
(tokenizer,7)
(namespace,9)
(wordindex,9)
(std,11)
(ostream,11)
(operator,11)
(std,11)
(ostream,11)
(os,11)
(tokenizer,11)
(value_type,11)
(const,11)
(value,11)
(return,13)
(os,13)
(value,13)
(first,13)
(value,13)
(second,13)
(void,18)
(copy_tokens,18)
(std,18)
(istream,18)
(is,18)
(std,18)
(ostream,18)
(os,18)
(tokenizer,20)
(tokenizer,20)
(is,20)
(std,22)
(copy,22)
(tokenizer,23)
(begin,23)
(input,23)
(collection,23)
(begin,23)
(tokenizer,24)
(end,24)
(input,24)
(collection,24)
(end,24)
(std,25)
(ostream_iterator,25)
(tokenizer,25)
(value_type,25)
(os,25)
(n,25)
(int,29)
(main,29)
(copy_tokens,31)
(std,31)
(cin,31)
(std,31)
(cout,31)


WordIndex: Storing Text and Linenumbers

Storing Text and Linenumbers
[On the Implementation]

WordIndex uses a word--linenumbers collection (a map) to store the words with the list of linenumbers where the word occurs.

The following is a simple way to insert a word--linenumber pair into a WordIndex collection.

 #include "WordIndex.h"

 void foo()
 {
    using wordindex::WordIndex;

    WordIndex map;
    map.insert( WordIndex::token_type( "myToken", 123 ) );
 }

Here's an example how tokens produced by class Tokenizer can be put in the word--linenumbers collection using an insert iterator. The insert iterator is created by the wordindex_inserter convenience function.

// g++ -I../../src -o insert_tokens.exe insert_tokens.cpp

#include <iostream>
#include <iterator>
#include "Tokenizer.h"
#include "WordIndex.h"

using wordindex::Tokenizer;
using wordindex::WordIndex;
using wordindex::wordindex_inserter;

void insert_tokens( std::istream& is, WordIndex& map )
{
   Tokenizer tokenizer( is );

   std::copy
   ( tokenizer.begin()              // input collection begin
   , tokenizer.end()                // input collection end
   , wordindex_inserter( map )      // convenience function that creates
   );                               //   the proper insert iterator
}

int main()
{
   WordIndex map;

   insert_tokens( std::cin, map );
}

EXPLAIN insert_iterator< WordIndex > wordindex_inserter (WordIndex &collection) convenience function to create WordIndex insert_iterator.


WordIndex: Unit Tests

Unit Tests

Detailed Description

For unit testing I used the Fructose testing framework, which is both versatile and easy to use.

Modules

Logger Unit Test
More description.
Pair Unit Test
More description.
Tokenizer Unit Test
More description.
Utility Unit Test
More description.
WordIndex Unit Test
More description.


WordIndex: Logger Unit Test

Logger Unit Test
[Unit Tests]

/*
 * Test-WordIndex.cpp - test WordIndex.
 */

// VC6: cannot compile
// VC7: cl -GX -GR -I ../include Test-WordIndex.cpp
// GCC: C:\Programs\MinGW\bin\g++.exe -I ../include -o Test-WordIndex.exe Test-WordIndex.cpp

#include "../src/Logger.h"
#include <Fructose/test_base.h>

struct test : public fructose::test_base< test >
{
   void is_proper_( const std::string& test_name )
   {
      fructose_assert( 0 && "implement" );
   }
};

int main( int argc, char* argv[] )
{
   test tests;
   tests.add_test( "is_proper_", &test::is_proper_ );

   return tests.run( argc, argv );
}

/*
 * end of file
 */


WordIndex: Pair Unit Test

Pair Unit Test
[Unit Tests]

/*
 * Test-Pair.cpp - test Pair.
 */

// VC6: cannot compile
// VC7: cl -GX -GR -I ../include Test-Pair.cpp
// GCC: C:\Programs\MinGW\bin\g++.exe -I ../include -o Test-Pair.exe Test-Pair.cpp

#include "../src/Pair.h"
#include <Fructose/test_base.h>

using wordindex::Pair;

struct test : public fructose::test_base< test >
{
   typedef int         first_type;
   typedef std::string second_type;

   typedef Pair< first_type, second_type > pair_type;

   first_type  first;
   second_type second;
   pair_type   pair;

   void setup()
   {
      first  = 33;
      second = "hello world";
      pair   = pair_type( first, second );
   }

   void teardown()
   {
      ; // do nothing
   }

   void is_proper_default_ctor( const std::string& test_name )
   {
      pair_type q;

      fructose_assert( first_type() == q.first && second_type() == q.second );
   }

   void is_proper_full_ctor( const std::string& test_name )
   {
      pair_type q( first, second );

      fructose_assert( first == q.first && second == q.second );
   }

   void is_proper_convert_from_first_ctor( const std::string& test_name )
   {
      pair_type q( first );
      // cannot convert from 'test::first_type' to 'wordindex::Pair<U,V>'
      // pair_type pair = first;

      fructose_assert( first == q.first && second_type() == q.second );
   }

   void is_proper_convert_from_second_ctor( const std::string& test_name )
   {
      pair_type q( second );
//      pair_type pair = second;

      fructose_assert( first_type() == q.first && second == q.second );
   }

   void is_proper_convert_from_first_assignment( const std::string& test_name )
   {
      pair_type q( second );

      q = first;

      fructose_assert( first == q.first && second == q.second );
   }

   void is_proper_convert_from_second_assignment( const std::string& test_name )
   {
      pair_type q( first );

      q = second;

      fructose_assert( first == q.first && second == q.second );
   }

   void is_proper_covert_to_first_operator( const std::string& test_name )
   {
      first_type x = pair;

      fructose_assert( first == x );
   }

   void is_proper_covert_to_second_operator( const std::string& test_name )
   {
      second_type x = pair;

      fructose_assert( second == x );
   }
};

int main( int argc, char* argv[] )
{
   test tests;
   tests.add_test( "is_proper_default_ctor"                  , &test::is_proper_default_ctor );
   tests.add_test( "is_proper_full_ctor"                     , &test::is_proper_full_ctor );
   tests.add_test( "is_proper_convert_from_first_ctor"       , &test::is_proper_convert_from_first_ctor );
   tests.add_test( "is_proper_convert_from_second_ctor"      , &test::is_proper_convert_from_second_ctor );
   tests.add_test( "is_proper_convert_from_first_assignment" , &test::is_proper_convert_from_first_assignment );
   tests.add_test( "is_proper_convert_from_second_assignment", &test::is_proper_convert_from_second_assignment );
   tests.add_test( "is_proper_covert_to_first_operator"      , &test::is_proper_covert_to_first_operator );
   tests.add_test( "is_proper_covert_to_second_operator"     , &test::is_proper_covert_to_second_operator );

   return tests.run( argc, argv );
}

/*
 * end of file
 */


WordIndex: Tokenizer Unit Test

Tokenizer Unit Test
[Unit Tests]

/*
 * Test-Tokenizer.cpp - test Tokenizer.
 */

// VC6: cannot compile
// VC7: cl -GX -GR -I ../include Test-Tokenizer.cpp
// GCC: C:\Programs\MinGW\bin\g++.exe -I ../include -o Test-Tokenizer.exe Test-Tokenizer.cpp

#include "../src/Tokenizer.h"
#include <Fructose/test_base.h>

using wordindex::Tokenizer;

const int number = 321;
const std::string text1 = "text";
const std::string text2 = "text2";
const std::string text3 = "_";
const std::string text4 = "_text";
const std::string text5 = "_text5";
const std::string text6 = "_123456";
const std::string text7 = "_1234text7";

struct test : public fructose::test_base< test >
{
   void is_working_iterator()
   {
   }

   void is_proper_text( const std::string& test_name )
   {
   }

   void is_proper_token_text( const std::string& test_name )
   {
      std::stringstream ss;

      ss <<
         number << " ~`!@# $%^ &*( ()-+= :;'\" ,./ <>? {}[]\\| " <<
         text1  << " " <<
         text2  << " " <<
         text3  << " " <<
         text4  << " " <<
         text5  << " " <<
         text6  << " " <<
         text7  << " " ;

      Tokenizer tokenizer( ss );

      Tokenizer::iterator pos( tokenizer.begin() );

      fructose_assert( ( pos != tokenizer.end() ) );

      fructose_assert( text1 == (*pos).first ); ++pos;
      fructose_assert( text2 == (*pos).first ); ++pos;
      fructose_assert( text3 == (*pos).first ); ++pos;
      fructose_assert( text4 == (*pos).first ); ++pos;
      fructose_assert( text5 == (*pos).first ); ++pos;
      fructose_assert( text6 == (*pos).first ); ++pos;
      fructose_assert( text7 == (*pos).first ); ++pos;

      fructose_assert( !( pos != tokenizer.end() ) );
   }
};

int main( int argc, char* argv[] )
{
   test tests;
   tests.add_test( "is_proper_token_text", &test::is_proper_token_text );

   return tests.run( argc, argv );
}

/*
 * end of file
 */


WordIndex: Utility Unit Test

Utility Unit Test
[Unit Tests]

/*
 * Test-WordIndex.cpp - test WordIndex.
 */

// VC6: cannot compile
// VC7: cl -GX -GR -I ../include Test-WordIndex.cpp
// GCC: C:\Programs\MinGW\bin\g++.exe -I ../include -o Test-WordIndex.exe Test-WordIndex.cpp

#include "../src/Utility.h"
#include <Fructose/test_base.h>

struct test : public fructose::test_base< test >
{
   void is_proper_( const std::string& test_name )
   {
      fructose_assert( 0 && "implement" );
   }
};

int main( int argc, char* argv[] )
{
   test tests;
   tests.add_test( "is_proper_", &test::is_proper_ );

   return tests.run( argc, argv );
}

/*
 * end of file
 */


WordIndex: WordIndex Unit Test

WordIndex Unit Test
[Unit Tests]

/*
 * Test-WordIndex.cpp - test WordIndex.
 */

// VC6: cannot compile
// VC7: cl -GX -GR -I ../include Test-WordIndex.cpp
// GCC: C:\Programs\MinGW\bin\g++.exe -I ../include -o Test-WordIndex.exe Test-WordIndex.cpp

#include "../src/WordIndex.h"
#include <Fructose/test_base.h>

using wordindex::WordIndex;

struct test : public fructose::test_base< test >
{
   void is_proper_( const std::string& test_name )
   {
      fructose_assert( 0 && "implement" );
   }
};

int main( int argc, char* argv[] )
{
   test tests;
   tests.add_test( "is_proper_", &test::is_proper_ );

   return tests.run( argc, argv );
}

/*
 * end of file
 */


WordIndex: Software Metrics

Software Metrics

Detailed Description

More description on MVG etc.

Modules

Software Metrics Overview
More description.
Software Metrics Per Function
More description.


WordIndex: Software Metrics Overview

Software Metrics Overview
[Software Metrics]


WordIndex: Software Metrics Per Function

Software Metrics Per Function
[Software Metrics]


WordIndex: Class List

  • Class List
  • Class Hierarchy
  • Class Members

WordIndex Class List

Here are the classes, structs, unions and interfaces with brief descriptions:
wordindex::AuthorVisitor | Handle --author option
wordindex::basic_tokenizer< C > | Separate stream input into token, line number pairs
wordindex::basic_tokenizer< C >::iterator | Iterator type
wordindex::ClapOutput | User defined output for the tclap commandline handling
wordindex::contained_in< C > | Value contained in collection predicate
wordindex::Context | Context for read and print
wordindex::FilenameConstraint | Tclap commandline filename constraint(s)
wordindex::insert_iterator< C > | Insert iterator
wordindex::Logger | Message logger class
wordindex::NoCaseLess | Std::string comparator for use in map
wordindex::Options | Program's options
wordindex::Pair< U, V > | Pair type with converting constructors and converting operators
wordindex::Printer | Function object to print an entry from the colleced words
wordindex::Reader | Process a file
wordindex::StringLess | Std::string comparator for use in map
wordindex::UnCopyable | Base class to implement UnCopyable interface
wordindex::WordIndex | Collect tokens with their associated line numbers


WordIndex: wordindex::AuthorVisitor Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::AuthorVisitor

wordindex::AuthorVisitor Class Reference

List of all members.

Detailed Description

handle --author option.

Definition at line 582 of file main.cpp.

Public Member Functions

AuthorVisitor (std::ostream *output, const std::string name)
constructor.
void | visit ()
handle this option (call-back).

Private Attributes

std::ostream * | m_output
the output stream
std::string | m_name
the author name
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::basic_tokenizer< C > Class Template Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::basic_tokenizer

wordindex::basic_tokenizer< C > Class Template Reference

#include <Tokenizer.h>

Inheritance diagram for wordindex::basic_tokenizer< C >:

wordindex::UnCopyable

List of all members.

Detailed Description

template<typename C>
class wordindex::basic_tokenizer< C >

separate stream input into token, line number pairs.

Definition at line 40 of file Tokenizer.h.

Public Types

typedef C | char_type
the character type.
typedef basic_tokenizer | class_type
the class type.
typedef int | line_number_type
the line number type.
typedef Pair< std::string,
line_number_type > 		value_type
the token--line number pair type.

Public Member Functions

basic_tokenizer (std::istream &is)
constructor.
iterator begin ()
return collection's begin iterator.
iterator end ()
return collection's end iterator.
void | set_skip_comments (const bool skip=true)
set or clear skip-comments flag.
void | add_to_start_set (std::string const &set)
add given set of characters to the the start symbol set.
void | add_to_followset (std::string const &set)
add given set of characters to the the follow symbol set.
void | set_lowercase (const bool lowercase)
set or clear the convert-to-lowercase flag.

Private Attributes

std::istream & | m_is
the input stream pointer; NULL if end-of-input has been reached.
bool | m_skip_comments
skip comments flag.
bool | m_lowercase
transform tokens to lowercase.
std::string | m_start
additional follow set.
std::string | m_follow
additional follow set.

Classes

class | iterator
the iterator type. More...
The documentation for this class was generated from the following file:
  • Tokenizer.h


WordIndex: wordindex::basic_tokenizer< C >::iterator Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::basic_tokenizer::iterator

wordindex::basic_tokenizer< C >::iterator Class Reference

#include <Tokenizer.h>

List of all members.

Detailed Description

template<typename C>
class wordindex::basic_tokenizer< C >::iterator

the iterator type.

Definition at line 66 of file Tokenizer.h.

Public Types

typedef iterator | class_type
the class type.

Public Member Functions

iterator (basic_tokenizer &tokenizer)
constructor.
iterator ()
default constructor.
const bool | is_start_comment (char_type chr)
true if character is a valid comment start character.
const bool | is_start (char_type chr)
true if character is a valid start character.
const bool | is_follow (char_type chr)
true if character is a valid second and following character.
const bool | is_start_or_follow (char_type chr)
true if character is a valid start or following character.
class_type & operator++ ()
advance to next token.
const value_type | operator * () const
return current token string.
const bool | operator!= (class_type const &rhs) const
true if this and other iterators are unequal.

Private Attributes

std::istream * | m_is
the input stream; NULL if end-of-input has been reached.
basic_tokenizer const * | m_tokenizer
skip comments flag.
std::string | m_token
current token string.
line_number_type m_line
current line number.

Member Function Documentation

template<typename C>
const value_type wordindex::basic_tokenizer< C >::iterator::operator * ( ) const [inline]

return current token string.

return current token string. return current token string, line number pair.

Definition at line 212 of file Tokenizer.h.

References wordindex::basic_tokenizer< C >::iterator::m_line, and wordindex::basic_tokenizer< C >::iterator::m_token.

The documentation for this class was generated from the following file:
  • Tokenizer.h


WordIndex: wordindex::ClapOutput Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::ClapOutput

wordindex::ClapOutput Class Reference

List of all members.

Detailed Description

user defined output for the tclap commandline handling.

Definition at line 551 of file main.cpp.

Public Member Functions

virtual void | failure (clp::CmdLineInterface &c, clp::ArgException &e)
fatal error, show help hint, terminate program.
virtual void | usage (clp::CmdLineInterface &c)
show program usage, terminate program.
virtual void | version (clp::CmdLineInterface &c)
show program version, terminate program.
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::contained_in< C > Class Template Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::contained_in

wordindex::contained_in< C > Class Template Reference

List of all members.

Detailed Description

template<typename C>
class wordindex::contained_in< C >

value contained in collection predicate.

Definition at line 344 of file main.cpp.

Public Types

typedef C | container_type
the set type.
typedef container_type::value_type | value_type
the value type.

Public Member Functions

contained_in (container_type const &c)
the value type.
bool | operator() (value_type const &value) const
the value type.

Private Attributes

container_type const & | m_container
the set.
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::Context Struct Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Context

wordindex::Context Struct Reference

List of all members.

Detailed Description

context for read and print.

Definition at line 284 of file main.cpp.

Public Attributes

Keywords keywords
the keywords specified
WordIndex wordindex
the non-keywords collected
The documentation for this struct was generated from the following file:
  • main.cpp


WordIndex: wordindex::FilenameConstraint Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::FilenameConstraint

wordindex::FilenameConstraint Class Reference

List of all members.

Detailed Description

tclap commandline filename constraint(s).

Definition at line 610 of file main.cpp.

Public Member Functions

FilenameConstraint (Logger &logger)
constructor.
virtual std::string | description () const
constraint description.
virtual std::string | shortID () const
constraint id.
virtual bool | check (const std::string &name) const
true if filename does not start with - or +.

Private Attributes

Logger & m_logger
logger to report through.
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::insert_iterator< C > Class Template Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::insert_iterator

wordindex::insert_iterator< C > Class Template Reference

#include <WordIndex.h>

List of all members.

Detailed Description

template<typename C>
class wordindex::insert_iterator< C >

insert iterator.

Definition at line 241 of file WordIndex.h.

Public Types

typedef C | container_type
the container type.
typedef insert_iterator | class_type
this class type.
typedef container_type::token_type | value_type
the value type.

Public Member Functions

insert_iterator (container_type &container)
constructor.
class_type & operator++ ()
pre-increment (no-op).
class_type & | operator++ (int)
post-increment (no-op).
class_type & operator * ()
de-reference.
class_type & | operator= (value_type const &value)
insert (add) a value.

Private Attributes

container_type & m_container
the container.
The documentation for this class was generated from the following file:
  • WordIndex.h


WordIndex: wordindex::Logger Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Logger

wordindex::Logger Class Reference

#include <Logger.h>

List of all members.

Detailed Description

message logger class.

Definition at line 40 of file Logger.h.

Public Member Functions

Logger (const std::string &progname, std::ostream &os=std::cout)
(default) constructor.
~Logger ()
destructor; close file, if required.
const int | GetLevel () const
the current logging level.
void | SetLevel (int const level)
set the loggin level.
void | Report (int const level, std::string const &msg)
report "programname: message" with given level.
void | Report (int const level, std::string const &filename, std::string const &msg)
report "filename: message" with given level.
void | ReportPart (int const level, std::string const &msg)
report "message" with given level.
void | Warning (std::string const &msg)
report a message with given level.
void | Fatal (std::string const &msg)
report a message with given level.

Private Attributes

int | m_level
the level to report.
std::ostream & | m_os
the output stream to write to.
std::string | m_progname
the program name.
The documentation for this class was generated from the following file:
  • Logger.h


WordIndex: wordindex::NoCaseLess Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::NoCaseLess

wordindex::NoCaseLess Class Reference

#include <WordIndex.h>

Inheritance diagram for wordindex::NoCaseLess:

wordindex::StringLess

List of all members.

Detailed Description

std::string comparator for use in map.

Definition at line 60 of file WordIndex.h.

Public Member Functions

virtual const bool | operator() (std::string const &s1, std::string const &s2) const
apply.

Static Private Member Functions

static const bool | nocase_less (char const c1, char const c2)
character comparator for std::lexicographical_compare().
The documentation for this class was generated from the following file:
  • WordIndex.h


WordIndex: wordindex::Options Struct Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Options

wordindex::Options Struct Reference

List of all members.

Detailed Description

program's options.

Definition at line 294 of file main.cpp.

Public Member Functions

Options ()
constructor.

Public Attributes

bool | frequency
report word usage percentage and count
bool | ignorecase
(currently not used)
bool | lowercase
convert words to lowercase
bool | reverse
only report keyword (stopword) usage
bool | summary
also report number of (key)words and references
int | name_width
name field width
The documentation for this struct was generated from the following file:
  • main.cpp


WordIndex: wordindex::Pair< U, V > Struct Template Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Pair

wordindex::Pair< U, V > Struct Template Reference

#include <Pair.h>

List of all members.

Detailed Description

template<typename U, typename V>
struct wordindex::Pair< U, V >

pair type with converting constructors and converting operators.

Definition at line 38 of file Pair.h.

Public Member Functions

Pair ()
default constructor.
Pair (U const &u, V const &v)
constructor.
Pair (U const &u)
construct from first type.
Pair (V const &v)
construct from second type.
Pair & | operator= (U const &u)
assign from first type.
Pair & | operator= (V const &v)
assign from second type.
operator const U () const
convert to first type.
operator const V () const
convert to second type.

Public Attributes

U | first
first object.
V | second
second object.
The documentation for this struct was generated from the following file:
  • Pair.h


WordIndex: wordindex::Printer Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Printer

wordindex::Printer Class Reference

List of all members.

Detailed Description

function object to print an entry from the colleced words.

Definition at line 465 of file main.cpp.

Public Member Functions

Printer (std::ostream &os, Options const &options, Context const &context)
constructor.
void | operator() (value_type const &value) const
print the given entry.

Private Types

typedef WordIndex::value_type | value_type
the value type.

Private Attributes

std::ostream & | m_os
the output stream.
Options const & | m_options
the options.
Context const & | m_context
the context (words and keywords).
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::Reader Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::Reader

wordindex::Reader Class Reference

List of all members.

Detailed Description

process a file.

Definition at line 416 of file main.cpp.

Public Types

typedef filename_list_element_type | value_type
the value type.

Public Member Functions

Reader (Options const &options, Context &context)
constructor.
void | operator() (value_type const &element)
read the given file.

Private Attributes

Options const & | m_options
the options.
Context & m_context
the context (words and keywords).
The documentation for this class was generated from the following file:
  • main.cpp


WordIndex: wordindex::StringLess Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::StringLess

wordindex::StringLess Class Reference

#include <WordIndex.h>

Inheritance diagram for wordindex::StringLess:

wordindex::NoCaseLess

List of all members.

Detailed Description

std::string comparator for use in map.

Definition at line 45 of file WordIndex.h.

Public Member Functions

virtual const bool operator() (std::string const &s1, std::string const &s2) const
apply.
The documentation for this class was generated from the following file:
  • WordIndex.h


WordIndex: wordindex::UnCopyable Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::UnCopyable

wordindex::UnCopyable Class Reference

#include <Utility.h>

Inheritance diagram for wordindex::UnCopyable:

wordindex::basic_tokenizer< C > wordindex::WordIndex

List of all members.

Detailed Description

base class to implement UnCopyable interface.

Definition at line 43 of file Utility.h.

Public Member Functions

UnCopyable ()
default constructor.

Private Types

typedef UnCopyable | class_type
this class type.

Private Member Functions

Not To Be Implemented
UnCopyable (class_type const &)
copy-assignment constructor.
class_type & | operator= (class_type const &)
copy-assignment operator.
The documentation for this class was generated from the following file:
  • Utility.h


WordIndex: wordindex::WordIndex Class Reference

  • Class List
  • Class Hierarchy
  • Class Members
wordindex::WordIndex

wordindex::WordIndex Class Reference

#include <WordIndex.h>

Inheritance diagram for wordindex::WordIndex:

wordindex::UnCopyable

List of all members.

Detailed Description

collect tokens with their associated line numbers.

Definition at line 89 of file WordIndex.h.

Public Types

typedef WordIndex | class_type
this class type.
typedef map_type::value_type | value_type
this value type.
typedef Pair< word_type,
line_number_type > 		token_type
the token--line number pair type.
typedef map_type::iterator | iterator
the string--line numbers map iterator type.
typedef map_type::const_iterator | const_iterator
the string--line numbers map const iterator type.

Public Member Functions

WordIndex ()
constructor.
const_iterator | begin () const
const begin iterator.
const_iterator | const_begin () const
const begin iterator.
const_iterator | end () const
const end iterator.
const_iterator | const_end () const
const end iterator.
void | insert (token_type const &pair)
add a token, line number pair.
void | insert (iterator pos, token_type const &pair)
add a token, line number pair.
void | insert (std::string const s, line_number_type const n)
add a token and line number.
const int | words () const
number of distinct words.
const int | lines () const
number of line references.

Private Types

typedef std::string | word_type
the word type.
typedef int | line_number_type
the line number type.
typedef std::vector
< line_number_type > 		locations_type
the list of line numbers type.
typedef std::map< word_type,
locations_type, StringLess > 		map_type
the token--locations associative array (map).

Private Attributes

int | m_lines
number of line references.
map_type m_words
datastructure: a map of token--list of linenumbers pairs (associative array).
The documentation for this class was generated from the following file:
  • WordIndex.h


WordIndex: Hierarchical Index

  • Class List
  • Class Hierarchy
  • Class Members

WordIndex Class Hierarchy

This inheritance list is sorted roughly, but not completely, alphabetically:
  • wordindex::AuthorVisitor
  • wordindex::basic_tokenizer< C >::iterator
  • wordindex::ClapOutput
  • wordindex::contained_in< C >
  • wordindex::Context
  • wordindex::FilenameConstraint
  • wordindex::insert_iterator< C >
  • wordindex::Logger
  • wordindex::Options
  • wordindex::Pair< U, V >
  • wordindex::Printer
  • wordindex::Reader
  • wordindex::StringLess
    • wordindex::NoCaseLess
  • wordindex::UnCopyable
    • wordindex::basic_tokenizer< C >
    • wordindex::WordIndex


WordIndex: Class Members

  • Class List
  • Class Hierarchy
  • Class Members
  • All
  • Functions
  • Variables
  • Typedefs
  • a
  • b
  • c
  • d
  • e
  • f
  • g
  • i
  • k
  • l
  • m
  • n
  • o
  • p
  • r
  • s
  • t
  • u
  • v
  • w
  • ~

Here is a list of all documented class members with links to the class documentation for each member:

- a -

  • add_to_followset() : wordindex::basic_tokenizer< C >
  • add_to_start_set() : wordindex::basic_tokenizer< C >
  • AuthorVisitor() : wordindex::AuthorVisitor

- b -

  • basic_tokenizer() : wordindex::basic_tokenizer< C >
  • begin()
: wordindex::WordIndex
, wordindex::basic_tokenizer< C >

- c -

  • char_type : wordindex::basic_tokenizer< C >
  • check() : wordindex::FilenameConstraint
  • class_type
: wordindex::basic_tokenizer< C >::iterator
, wordindex::UnCopyable
, wordindex::basic_tokenizer< C >
, wordindex::WordIndex
, wordindex::insert_iterator< C >
  • const_begin() : wordindex::WordIndex
  • const_end() : wordindex::WordIndex
  • const_iterator : wordindex::WordIndex
  • contained_in() : wordindex::contained_in< C >
  • container_type : wordindex::insert_iterator< C > , wordindex::contained_in< C >

- d -

  • description() : wordindex::FilenameConstraint

- e -

  • end()
: wordindex::basic_tokenizer< C >
, wordindex::WordIndex

- f -

  • failure() : wordindex::ClapOutput
  • Fatal() : wordindex::Logger
  • FilenameConstraint() : wordindex::FilenameConstraint
  • first : wordindex::Pair< U, V >
  • frequency : wordindex::Options

- g -

  • GetLevel() : wordindex::Logger

- i -

  • ignorecase : wordindex::Options
  • insert() : wordindex::WordIndex
  • insert_iterator() : wordindex::insert_iterator< C >
  • is_follow() : wordindex::basic_tokenizer< C >::iterator
  • is_start() : wordindex::basic_tokenizer< C >::iterator
  • is_start_comment() : wordindex::basic_tokenizer< C >::iterator
  • is_start_or_follow() : wordindex::basic_tokenizer< C >::iterator
  • iterator : wordindex::WordIndex , wordindex::basic_tokenizer< C >::iterator

- k -

  • keywords : wordindex::Context

- l -

  • line_number_type : wordindex::basic_tokenizer< C > , wordindex::WordIndex
  • lines() : wordindex::WordIndex
  • locations_type : wordindex::WordIndex
  • Logger() : wordindex::Logger
  • lowercase : wordindex::Options

- m -

  • m_container : wordindex::contained_in< C > , wordindex::insert_iterator< C >
  • m_context : wordindex::Printer , wordindex::Reader
  • m_follow : wordindex::basic_tokenizer< C >
  • m_is : wordindex::basic_tokenizer< C >::iterator , wordindex::basic_tokenizer< C >
  • m_level : wordindex::Logger
  • m_line : wordindex::basic_tokenizer< C >::iterator
  • m_lines : wordindex::WordIndex
  • m_logger : wordindex::FilenameConstraint
  • m_lowercase : wordindex::basic_tokenizer< C >
  • m_name : wordindex::AuthorVisitor
  • m_options : wordindex::Printer , wordindex::Reader
  • m_os : wordindex::Logger , wordindex::Printer
  • m_output : wordindex::AuthorVisitor
  • m_progname : wordindex::Logger
  • m_skip_comments : wordindex::basic_tokenizer< C >
  • m_start : wordindex::basic_tokenizer< C >
  • m_token : wordindex::basic_tokenizer< C >::iterator
  • m_tokenizer : wordindex::basic_tokenizer< C >::iterator
  • m_words : wordindex::WordIndex
  • map_type : wordindex::WordIndex

- n -

  • name_width : wordindex::Options
  • nocase_less() : wordindex::NoCaseLess

- o -

  • operator *() : wordindex::basic_tokenizer< C >::iterator , wordindex::insert_iterator< C >
  • operator const U() : wordindex::Pair< U, V >
  • operator const V() : wordindex::Pair< U, V >
  • operator!=() : wordindex::basic_tokenizer< C >::iterator
  • operator()()
: wordindex::Printer
, wordindex::StringLess
, wordindex::NoCaseLess
, wordindex::contained_in< C >
, wordindex::Reader
  • operator++()
: wordindex::insert_iterator< C >
, wordindex::basic_tokenizer< C >::iterator
, wordindex::insert_iterator< C >
  • operator=()
: wordindex::UnCopyable
, wordindex::Pair< U, V >
, wordindex::insert_iterator< C >
  • Options() : wordindex::Options

- p -

  • Pair() : wordindex::Pair< U, V >
  • Printer() : wordindex::Printer

- r -

  • Reader() : wordindex::Reader
  • Report() : wordindex::Logger
  • ReportPart() : wordindex::Logger
  • reverse : wordindex::Options

- s -

  • second : wordindex::Pair< U, V >
  • set_lowercase() : wordindex::basic_tokenizer< C >
  • set_skip_comments() : wordindex::basic_tokenizer< C >
  • SetLevel() : wordindex::Logger
  • shortID() : wordindex::FilenameConstraint
  • summary : wordindex::Options

- t -

  • token_type : wordindex::WordIndex

- u -

  • UnCopyable() : wordindex::UnCopyable
  • usage() : wordindex::ClapOutput

- v -

  • value_type
: wordindex::contained_in< C >
, wordindex::Reader
, wordindex::basic_tokenizer< C >
, wordindex::insert_iterator< C >
, wordindex::WordIndex
, wordindex::Printer
  • version() : wordindex::ClapOutput
  • visit() : wordindex::AuthorVisitor

- w -

  • Warning() : wordindex::Logger
  • word_type : wordindex::WordIndex
  • WordIndex() : wordindex::WordIndex
  • wordindex : wordindex::Context
  • words() : wordindex::WordIndex

- ~ -

  • ~Logger() : wordindex::Logger


WordIndex: Namespace Index

  • Namespace List
  • Namespace Members

WordIndex Namespace List

Here is a list of all documented namespaces with brief descriptions:
wordindex The application's namespace


WordIndex: wordindex Namespace Reference

  • Namespace List
  • Namespace Members

wordindex Namespace Reference

Detailed Description

The application's namespace.

clp namespace shorthand for TCLAP.

namespace for this application.

Classes

class | Logger
message logger class. More...
struct | Context
context for read and print. More...
struct | Options
program's options. More...
class | contained_in
value contained in collection predicate. More...
class | Reader
process a file. More...
class | Printer
function object to print an entry from the colleced words. More...
class | ClapOutput
user defined output for the tclap commandline handling. More...
class | AuthorVisitor
handle --author option. More...
class | FilenameConstraint
tclap commandline filename constraint(s). More...
struct | Pair
pair type with converting constructors and converting operators. More...
class | basic_tokenizer
separate stream input into token, line number pairs. More...
class | UnCopyable
base class to implement UnCopyable interface. More...
class | StringLess
std::string comparator for use in map. More...
class | NoCaseLess
std::string comparator for use in map. More...
class | WordIndex
collect tokens with their associated line numbers. More...
class | insert_iterator
insert iterator. More...

Typedefs

typedef int | line_number_type
the line number type.
typedef std::string | word_type
the word type.
typedef std::string | filename_type
the filename type.
typedef std::set< word_type > | keyword_collection_type
the keyword container.
typedef keyword_collection_type | Keywords
the keyword container.
typedef Pair< filename_type,
line_number_type > 		filename_list_element_type
the filename list element type.
typedef std::vector
< filename_list_element_type > 		filename_list_type
the filename list type.
typedef basic_tokenizer< char > | Tokenizer
tokenizer for char character type.

Functions

std::string const | try_help ("Try '"+program_name+" --help' for more information.\n")
hint to read help.
template<typename C>
void | print_collection (std::ostream &os, C const &collection)
print collections contents.
int | usage (std::ostream &os, int status)
the program usage.
int | version (std::ostream &os, int status)
the version help.
void | check_file_exists (filename_list_element_type const &element)
filename list entry exists predicate.
void | read (std::istream &is, Options const &options, Context &context)
read words from the given stream into the wordindex.
void | print (std::ostream &os, Options const &options, Context const &context)
print the collected words.
const filename_list_element_type | to_filename_list_element (filename_type const &filename)
convert a filename to a filename list element.
int | main (int argc, char *argv[])
the program.
const char * | to_charptr (const std::string &s)
string to character pointer conversion shim.
const std::string | to_string (long const x)
integer to string conversion shim.
const std::string | to_string (int const x)
integer to string conversion shim.
const std::string | to_string (double const x, int const n=3)
integer to string conversion shim.
const std::string | to_version_string (int const version, int const div)
convert version to string.
const long | to_int (const char *s)
string to integer conversion shim.
const double | to_real (const char *s)
string to real conversion shim.
const std::string | basename (std::string const filename)
the filename without the extension.
const std::string | filename (std::string const filename)
the filename without the directory part and extension.
const std::string | extension (std::string const filename)
the filename extension.
const bool | exist (std::string const filename)
true if the specified file is "-" (stdin) or exists as diskfile, false otherwise.
insert_iterator< WordIndex > | wordindex_inserter (WordIndex &collection)
convenience function to create WordIndex insert_iterator.
Configuration
std::string const | author_string ("Martin J. Moene <m.j.moene@eld.physics.LeidenUniv.nl>")
the author name.
std::string const | version_string (WORDINDEX_RELEASE_NUMBER_STRING)
the program version.
std::string const | date_string (WORDINDEX_RELEASE_DATE_STRING)
the program date.
std::string | program_name ("wordindex")
the program name / basename.
std::string const | banner (program_name+" "+version_string+" ("+date_string+") create alphabetically sorted index of words.\n\n")
the program banner.
std::string const | copyright ("Copyright 2007-2008, by Martin J. Moene.\n")
the program copyright.

Variables

Logger | logger (program_name, std::cout)
the logger instance (external linkage).


WordIndex: Class Members

  • Namespace List
  • Namespace Members
  • All
  • Functions
  • Variables
  • Typedefs
  • a
  • b
  • c
  • d
  • e
  • f
  • k
  • l
  • m
  • p
  • r
  • t
  • u
  • v
  • w

Here is a list of all documented namespace members with links to the namespaces they belong to:

- a -

  • author_string() : wordindex

- b -

  • banner() : wordindex
  • basename() : wordindex

- c -

  • check_file_exists() : wordindex
  • copyright() : wordindex

- d -

  • date_string() : wordindex

- e -

  • exist() : wordindex
  • extension() : wordindex

- f -

  • filename() : wordindex
  • filename_list_element_type : wordindex
  • filename_list_type : wordindex
  • filename_type : wordindex

- k -

  • keyword_collection_type : wordindex
  • Keywords : wordindex

- l -

  • line_number_type : wordindex
  • logger : wordindex

- m -

  • main() : wordindex

- p -

  • print() : wordindex
  • print_collection() : wordindex
  • program_name() : wordindex

- r -

  • read() : wordindex

- t -

  • to_charptr() : wordindex
  • to_filename_list_element() : wordindex
  • to_int() : wordindex
  • to_real() : wordindex
  • to_string() : wordindex
  • to_version_string() : wordindex
  • Tokenizer : wordindex
  • try_help() : wordindex

- u -

  • usage() : wordindex

- v -

  • version() : wordindex
  • version_string() : wordindex

- w -

  • word_type : wordindex
  • wordindex_inserter() : wordindex


WordIndex: File Index

  • File List
  • File Members

WordIndex File List

Here is a list of all documented files with brief descriptions:
00_template.cpp [code]
00_template.h [code]
Config.h [code]
Logger.h [code]
main.cpp [code]
Pair.h [code]
Tokenizer.h [code]
Utility.h [code]
version.h [code]
WordIndex.h [code]


WordIndex: Config.h File Reference

Config.h File Reference

Detailed Description

Definition in file Config.h.

Go to the source code of this file.

Defines

#define typename_type_k typename
compiler selection.

Define Documentation

#define typename_type_k typename

compiler selection.

compiler dependent typename behaviour.

Definition at line 55 of file Config.h.


WordIndex: Logger.h File Reference

Logger.h File Reference

Detailed Description

Definition in file Logger.h.

#include <iostream>
#include <string>

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

class | wordindex::Logger
message logger class. More...


WordIndex: main.cpp File Reference

main.cpp File Reference

Detailed Description

Definition in file main.cpp.

#include "Config.h"
#include "Logger.h"
#include "Pair.h"
#include "Tokenizer.h"
#include "Utility.h"
#include "Version.h"
#include "WordIndex.h"
#include <ctype.h>
#include <algorithm>
#include <set>
#include <iterator>
#include <iomanip>
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <tclap/CmdLine.h>

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

struct | wordindex::Context
context for read and print. More...
struct | wordindex::Options
program's options. More...
class | wordindex::contained_in< C >
value contained in collection predicate. More...
class | wordindex::Reader
process a file. More...
class | wordindex::Printer
function object to print an entry from the colleced words. More...
class | wordindex::ClapOutput
user defined output for the tclap commandline handling. More...
class | wordindex::AuthorVisitor
handle --author option. More...
class | wordindex::FilenameConstraint
tclap commandline filename constraint(s). More...

Defines

#define | dimensionof(a) ( sizeof(a) / sizeof( *(a) ) )
the dimension of an array.

Typedefs

typedef int | wordindex::line_number_type
the line number type.
typedef std::string | wordindex::word_type
the word type.
typedef std::string | wordindex::filename_type
the filename type.
typedef std::set< word_type > | wordindex::keyword_collection_type
the keyword container.
typedef keyword_collection_type | wordindex::Keywords
the keyword container.
typedef Pair< filename_type,
line_number_type > 		wordindex::filename_list_element_type
the filename list element type.
typedef std::vector
< filename_list_element_type > 		wordindex::filename_list_type
the filename list type.

Functions

std::string const | wordindex::try_help ("Try '"+program_name+" --help' for more information.\n")
hint to read help.
template<typename C>
void | wordindex::print_collection (std::ostream &os, C const &collection)
print collections contents.
int | wordindex::usage (std::ostream &os, int status)
the program usage.
int | wordindex::version (std::ostream &os, int status)
the version help.
void | wordindex::check_file_exists (filename_list_element_type const &element)
filename list entry exists predicate.
void | wordindex::read (std::istream &is, Options const &options, Context &context)
read words from the given stream into the wordindex.
void | wordindex::print (std::ostream &os, Options const &options, Context const &context)
print the collected words.
const filename_list_element_type | wordindex::to_filename_list_element (filename_type const &filename)
convert a filename to a filename list element.
int | wordindex::main (int argc, char *argv[])
the program.
int | main (int argc, char *argv[])
the program.
Configuration
std::string const | wordindex::author_string ("Martin J. Moene <m.j.moene@eld.physics.LeidenUniv.nl>")
the author name.
std::string const | wordindex::version_string (WORDINDEX_RELEASE_NUMBER_STRING)
the program version.
std::string const | wordindex::date_string (WORDINDEX_RELEASE_DATE_STRING)
the program date.
std::string | wordindex::program_name ("wordindex")
the program name / basename.
std::string const | wordindex::banner (program_name+" "+version_string+" ("+date_string+") create alphabetically sorted index of words.\n\n")
the program banner.
std::string const | wordindex::copyright ("Copyright 2007-2008, by Martin J. Moene.\n")
the program copyright.

Variables

Logger | wordindex::logger (program_name, std::cout)
the logger instance (external linkage).


WordIndex: Pair.h File Reference

Pair.h File Reference

Detailed Description

Definition in file Pair.h.

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

struct | wordindex::Pair< U, V >
pair type with converting constructors and converting operators. More...


WordIndex: Tokenizer.h File Reference

Tokenizer.h File Reference

Detailed Description

Definition in file Tokenizer.h.

#include "Pair.h"
#include "Utility.h"
#include <cassert>

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

class | wordindex::basic_tokenizer< C >
separate stream input into token, line number pairs. More...
class | wordindex::basic_tokenizer< C >::iterator
the iterator type. More...

Typedefs

typedef basic_tokenizer< char > | wordindex::Tokenizer
tokenizer for char character type.


WordIndex: Utility.h File Reference

Utility.h File Reference

Detailed Description

Definition in file Utility.h.

#include "Config.h"
#include <io.h>
#include <string>
#include <sstream>

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

class | wordindex::UnCopyable
base class to implement UnCopyable interface. More...

Functions

const char * | wordindex::to_charptr (const std::string &s)
string to character pointer conversion shim.
const std::string | wordindex::to_string (long const x)
integer to string conversion shim.
const std::string | wordindex::to_string (int const x)
integer to string conversion shim.
const std::string | wordindex::to_string (double const x, int const n=3)
integer to string conversion shim.
const std::string | wordindex::to_version_string (int const version, int const div)
convert version to string.
const long | wordindex::to_int (const char *s)
string to integer conversion shim.
const double | wordindex::to_real (const char *s)
string to real conversion shim.
const std::string | wordindex::basename (std::string const filename)
the filename without the extension.
const std::string | wordindex::filename (std::string const filename)
the filename without the directory part and extension.
const std::string | wordindex::extension (std::string const filename)
the filename extension.
const bool | wordindex::exist (std::string const filename)
true if the specified file is "-" (stdin) or exists as diskfile, false otherwise.


WordIndex: version.h File Reference

version.h File Reference

Detailed Description

Definition in file version.h.

Go to the source code of this file.

Defines

#define | WORDINDEX_RELEASE_NUMBER_STRING "0.0.2"
version number as string
#define | WORDINDEX_RELEASE_DATE_STRING "08 Feb 2008"
release date as string


WordIndex: WordIndex.h File Reference

WordIndex.h File Reference

Detailed Description

Definition in file WordIndex.h.

#include "Pair.h"
#include "Utility.h"
#include <map>
#include <vector>
#include <string>

Go to the source code of this file.

Namespaces

namespace | wordindex

Classes

class | wordindex::StringLess
std::string comparator for use in map. More...
class | wordindex::NoCaseLess
std::string comparator for use in map. More...
class | wordindex::WordIndex
collect tokens with their associated line numbers. More...
class | wordindex::insert_iterator< C >
insert iterator. More...

Functions

insert_iterator< WordIndex > | wordindex::wordindex_inserter (WordIndex &collection)
convenience function to create WordIndex insert_iterator.


WordIndex: Class Members

  • File List
  • File Members
  • All
  • Functions
  • Defines
Here is a list of all documented file members with links to the documentation:

  • dimensionof : main.cpp
  • main() : main.cpp
  • typename_type_k : Config.h
  • WORDINDEX_RELEASE_DATE_STRING : version.h
  • WORDINDEX_RELEASE_NUMBER_STRING : version.h


WordIndex: Page Index

WordIndex Related Pages

Here is a list of all related documentation pages:
  • References
  • Software


WordIndex: References

References

[Aho et al., 1986]
Alfred V. Aho, Ravi Sethi, and Jeffrey Ullman. Compilers -- Principles, Techniques and Tools. 1986. Addison Wesley. ISBN 0-201-10088-6. Also known as the dragon book.

[Alexandrescu, 2001]
Andrei Alexandrescu. Modern C++ Design: Generic Programming and Design Patterns Applied. Addison–Wesley, 2001. ISBN 0201704315.

[Feathers, 2004]
Michael Feathers. Working Effectively with Legacy Code. Prentice Hall, September 2004. ISBN-10: 0–13–117705–2, ISBN-13: 978–0--13–117705–5.

[Fowler, 1999]
Martin Fowler. Refactoring: Improving the Design of Existing Code. Addison–Wesley Professional, 1st edition, June 1999. ISBN-10 0–201–48567–2, ISBN-13 978–0--201–48567–7.

[Gamma et al., 1995]
Erick Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns; Elements of Reusable Object-Oriented Software. Addison–Wesley, 1995. ISBN 0–201–63361–2.

[Josuttis, 1999]
Nicolai M. Josuttis. The C++ Standard Library: A Tutorial and Reference. Addison–Wesley, 1999. ISBN 0–201–37926–0.

[Kerievsky, 2004]
Joshua Kerievsky. Refactoring to Patterns. Addison–Wesley Professional, August 2004. ISBN-10 0321213351, ISBN-13 978-0321213358.

[Kernighan and Pike, 1984]
Brian W. Kernighan and Rob Pike. The UNIX programming environment. Prentice Hall, 1984. ISBN 0–13–937681–X.

[Kernighan and Pike, 1999]
Brian W. Kernighan and Rob Pike. The Practice of Programming. Addison–Wesley, 1999. ISBN 0–201–61586–X.

[Main, 2007]
Chris Main. C++ Unit Test Frameworks. in ACCU Overload #78 - Apr 2007.

[Martin, 2003]
Robert C. Martin. Agile Software Development, Principles, Patterns, and Practices. Prentice Hall, 2003. ISBN 0–13–597444–5.

[Miller, 1997]
Peter Miller. Recursive Make Considered Harmful.

[Meyers, 2005]
Scott Meyers. Effective C++: 55 Specific Ways to Improve Your Programs and Designs. Addison–Wesley, 3rd edition, 2005. ISBN 0-321-33487-6.

[Moene, 2004]
Martin Moene. A proposal for software revision control at ELD.

[Spinellis, 2003]
Diomidis Spinellis. Code Reading: The Open Source Perspective. Addison–Wesley, 2003. ISBN 0–201–79940–5.

[Spinellis, 2006]
Diomidis Spinellis. Code Quality: The Open Source Perspective. Addison–Wesley, 2006. ISBN 0–321–16607–8.

[Stroustrup, 1997]
Bjarne Stroustrup. The C++ Programming Language. Addison–Wesley, third edition, 1997. ISBN 0–201–88954–4.

[Vandevoorde and Josuttis, 2002]
David Vandevoorde and Nicolai M. Josuttis. C++ Templates - The Complete Guide. Addison–Wesley, 2002. ISBN 0–201–73484–2.

[Software Metric, Wikipedia]
Software metric. (2007, December 24). In Wikipedia, The Free Encyclopedia. Retrieved 13:46, December 29, 2007, from http://en.wikipedia.org/w/index.php?title=Software_metric&oldid=179967990

[STL, Wikipedia]
Standard Template Library. (2007, November 18). In Wikipedia, The Free Encyclopedia. Retrieved 13:51, December 29, 2007, from http://en.wikipedia.org/w/index.php?title=Standard_Template_Library&oldid=172238817

[Wilson, 2005]
Matthew Wilson. Imperfect C++: Practical Solutions for Real-Life Programming. Addison–Wesley Professional, 2005. ISBN 0–321–22877–4.

[Wilson, 2007]
Matthew Wilson. Extended STL, Volume 1: Collections and Iterators. Addison–Wesley Professional, 2007. ISBN-10 0–321–30550–7, ISBN-13 978–0--321–30550–3.


WordIndex: Software

Software

[AntConc]
AntConc, a freeware concordance program for Windows, Macintosh OS X, and Linux.

[BDS C]
BDS C. (2007, October 21). In Wikipedia, The Free Encyclopedia. Retrieved 20:08, January 7, 2008, from http://en.wikipedia.org/w/index.php?title=BDS_C&oldid=166135994

[Bondwell 12]
Bondwell. (2007, May 1). In Wikipedia, The Free Encyclopedia. Retrieved 19:58, January 7, 2008, from http://en.wikipedia.org/w/index.php?title=Bondwell&oldid=127379329

[CCCC]
CCCC - C and C++ Code Counter.

[Code::Blocks IDE]
Code::Blocks IDE.

[Concordance]
Concordance, text analysis and concordance software.

[Context]
Context Programmers Editor.

[CP/M]
CP/M. (2008, January 5). In Wikipedia, The Free Encyclopedia. Retrieved 20:10, January 7, 2008, from http://en.wikipedia.org/w/index.php?title=CP/M&oldid=182406242

[Doxygen]
Doxygen, a documentation system for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors) and to some extent PHP, C#, and D..

[FastDep]
FastDep, a fast dependency generator for C/C++ files.

[FRUCTOSE C++ Unit Testing Framework]
FRUCTOSE - Andrew Marlow's C++ Unit Testing Framework.

[Inno Setup]
Inno Setup, a free installer for Windows programs.

[GNU C]
GCC, the GNU Compiler Collection..

[GNU Style and Diction]
GNU Style and Diction.

[LocMetrix]
locmetrics.com.

[Microsoft Visual C++]
Visual Studio, a range of tools that offer many benefits for individual developers and software development teams..

[Microsoft HTML Help Compiler]
HTML Help.

[Subversion]
Subversion, a compelling replacement for CVS.

[Templatized C++ Command Line Parser Library]
TCLAP, Templatized C++ Command Line Parser Library by Michael E. Smoot.

[TortoiseSVN]
TortoiseSVN, a Subversion client, implemented as a windows shell extension.

[Refactor! for C++]
Refactor! for C++.

[Visual Assist X]
Visual Assist X.

[xref Cross-Reference Program]
xref - Martin Moene's cross-reference program.

[WordStar]
WordStar. (2008, January 2). In Wikipedia, The Free Encyclopedia. Retrieved 20:11, January 7, 2008, from http://en.wikipedia.org/w/index.php?title=WordStar&oldid=181739927


WordIndex: 00_template.cpp Source File

00_template.cpp

00001 /*
00002  * @name@.cpp - DLL.
00003  *
00004  * This file is part of Concordance.
00005  *
00006  * Copyright (C) 2007, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: 00_template.cpp 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 #include <hdl/mids.h>
00026
00027 /*
00028  * end of file
00029  */


WordIndex: 00_template.h Source File

00_template.h

00001 /*
00002  * @name@.h - .
00003  *
00004  * This file is part of Concordance.
00005  *
00006  * Copyright (C) 2007, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: 00_template.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 #ifndef @name@_h_included
00026 #define @name@_h_included
00027
00028 #endif // @name@_h_included
00029
00030 /*
00031  * end of file
00032  */


WordIndex: Config.h Source File

Config.h

Go to the documentation of this file.
00001 /*
00002  * config.h - compiler dependent issues.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: Config.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef config_h_included
00030 #define config_h_included
00031
00032 /**
00033  * compiler selection.
00034  */
00035 #if defined( _MSC_VER )
00036 # define _WORDINDEX_MSC
00037 # if _MSC_VER < 1310
00038 #  define _WORDINDEX_MSC6
00039 # endif
00040
00041 #elif defined ( __INTEL_COMPILER )
00042 # define _WORDINDEX_INTEL
00043
00044 #elif defined ( __GNUC__ )
00045 # define _WORDINDEX_GNU
00046
00047 #endif
00048
00049 /**
00050  * compiler dependent typename behaviour.
00051  */
00052 #if defined( _WORDINDEX_MSC6 )
00053 # define typename_type_k
00054 #else
00055 # define typename_type_k typename
00056 #endif
00057
00058 #if defined ( _WORDINDEX_MSC6 )
00059 namespace std {
00060 template < typename T >
00061 inline T min( T const& a, T const& b ) { return a < b ? a : b; }
00062
00063 template < typename T >
00064 inline T max( T const& a, T const& b ) { return a > b ? a : b; }
00065 } // namespace std
00066 #endif
00067
00068 #endif // config_h_included
00069
00070 /*
00071  * end of file
00072  */


WordIndex: Logger.h Source File

Logger.h

Go to the documentation of this file.
00001 /*
00002  * src/Logger.h - message, warning and error logger class.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: Logger.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef logger_h_in_included
00030 #define logger_h_in_included
00031
00032 #include <iostream>  // for std::cerr, std::ostream
00033 #include <string>    // for std::string
00034
00035 namespace wordindex {
00036
00037 /**
00038  * message logger class.
00039  */
00040 class Logger
00041 {
00042 public:
00043    /**
00044     * (default) constructor.
00045     */
00046    Logger( const std::string& progname, std::ostream& os = std::cout )
00047       : m_level   ( 0  )
00048       , m_os      ( os )
00049       , m_progname( progname )
00050    {
00051    }
00052
00053    /**
00054     * destructor; close file, if required.
00055     */
00056    ~Logger()
00057    {
00058    }
00059
00060    /**
00061     * the current logging level.
00062     */
00063    const int GetLevel() const
00064    {
00065       return m_level;
00066    }
00067
00068    /**
00069     * set the loggin level.
00070     */
00071    void SetLevel( int const level )
00072    {
00073       m_level = level;
00074    }
00075
00076    /**
00077     * report "programname: message" with given level.
00078     */
00079    void Report( int const level, std::string const& msg )
00080    {
00081       Report( level, m_progname, msg );
00082    }
00083
00084    /**
00085     * report "filename: message" with given level.
00086     */
00087    void Report( int const level, std::string const& filename, std::string const& msg )
00088    {
00089       if ( level <= m_level )
00090       {
00091          m_os << filename << ": " << msg;
00092       }
00093    }
00094
00095    /**
00096     * report "message" with given level.
00097     */
00098    void ReportPart( int const level, std::string const& msg )
00099    {
00100       if ( level <= m_level )
00101       {
00102          m_os << msg;
00103       }
00104    }
00105
00106    /**
00107     * report a message with given level.
00108     */
00109    void Warning( std::string const& msg )
00110    {
00111       if ( &m_os != &std::cout
00112       &&   &m_os != &std::cerr
00113       &&   &m_os != &std::clog
00114       )
00115       {
00116          Report( 0, "Warning: " + msg );
00117       }
00118       std::cerr << m_progname << ": Warning: " << msg << std::endl;
00119    }
00120
00121    /**
00122     * report a message with given level.
00123     */
00124    void Fatal( std::string const& msg )
00125    {
00126       if ( &m_os != &std::cout
00127       &&   &m_os != &std::cerr
00128       &&   &m_os != &std::clog
00129       )
00130       {
00131          Report( 0, msg );
00132       }
00133       std::cerr << m_progname << ": " << msg << std::endl;
00134       exit( 1 );
00135    }
00136
00137 private:
00138    /**
00139     * the level to report.
00140     */
00141    int m_level;
00142
00143    /**
00144     * the output stream to write to.
00145     */
00146    std::ostream& m_os;
00147
00148    /**
00149     * the program name.
00150     */
00151    std::string m_progname;
00152 };
00153
00154 } // namespace wordindex
00155
00156 #endif //#ifndef logger_h_in_included
00157
00158 /*
00159  * end of file
00160  */
00161


WordIndex: Pair.h Source File

Pair.h

Go to the documentation of this file.
00001  /*
00002  * Pair.h - token, line number pair type.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: Pair.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef _pair_h_included
00030 #define _pair_h_included
00031
00032 namespace wordindex {
00033
00034 /**
00035  * pair type with converting constructors and converting operators.
00036  */
00037 template < typename U, typename V >
00038 struct Pair
00039 {
00040    /**
00041     * default constructor.
00042     */
00043    Pair() : first(), second() { ; }
00044
00045    /**
00046     * constructor.
00047     */
00048    Pair( U const& u, V const& v ) : first( u ), second( v ) { ; }
00049
00050    /**
00051     * construct from first type.
00052     */
00053    explicit Pair( U const& u ) : first( u ), second() { ; }
00054
00055    /**
00056     * construct from second type.
00057     */
00058    explicit Pair( V const& v ) : first(), second( v ) { ; }
00059
00060    /**
00061     * assign from first type.
00062     */
00063    Pair& operator=( U const& u )
00064    {
00065       first = u;
00066       return *this;
00067    }
00068
00069    /**
00070     * assign from second type.
00071     */
00072    Pair& operator=( V const& v )
00073    {
00074       second = v;
00075       return *this;
00076    }
00077
00078    /**
00079     * convert to first type.
00080     */
00081    operator const U() const
00082    {
00083       return first;
00084    }
00085
00086    /**
00087     * convert to second type.
00088     */
00089    operator const V() const
00090    {
00091       return second;
00092    }
00093
00094    /**
00095     * first object.
00096     */
00097    U first;
00098
00099    /**
00100     * second object.
00101     */
00102    V second;
00103 };
00104
00105 } // namespace wordindex
00106
00107 #endif // _pair_h_included
00108
00109 /*
00110  * end of file
00111  */
00112


WordIndex: Tokenizer.h Source File

Tokenizer.h

Go to the documentation of this file.
00001  /*
00002  * Tokenizer.h - tokenizer collection.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: Tokenizer.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #include "Pair.h"     // for pair_type
00030 #include "Utility.h"  // for class UnCopyable
00031
00032 #include <cassert>     // for ::assert()
00033
00034 namespace wordindex {
00035
00036 /**
00037  * separate stream input into token, line number pairs.
00038  */
00039 template < typename C >
00040 class basic_tokenizer : private UnCopyable
00041 {
00042 public:
00043    /**
00044     * the character type.
00045     */
00046    typedef C char_type;
00047
00048    /**
00049     * the class type.
00050     */
00051    typedef basic_tokenizer class_type;
00052
00053    /**
00054     * the line number type.
00055     */
00056    typedef int line_number_type;
00057
00058    /**
00059     * the token--line number pair type.
00060     */
00061    typedef Pair< std::string, line_number_type > value_type;
00062
00063    /**
00064     * the iterator type.
00065     */
00066    class iterator : public std::iterator< std::input_iterator_tag, value_type >
00067    {
00068    public:
00069       /**
00070        * the class type.
00071        */
00072       typedef iterator class_type;
00073
00074       /**
00075        * constructor.
00076        */
00077       explicit iterator( basic_tokenizer& tokenizer )
00078       : m_is       ( &tokenizer.m_is )
00079       , m_tokenizer( &tokenizer      )
00080       , m_line     (  1 )
00081       {
00082          operator++();
00083       }
00084
00085       /**
00086        * default constructor.
00087        */
00088       iterator()
00089       : m_is       ( NULL )
00090       , m_tokenizer( NULL )
00091       , m_line     (  0   )
00092       {
00093       }
00094
00095       /**
00096        * true if character is a valid comment start character.
00097        */
00098       const bool is_start_comment( char_type chr )
00099       {
00100           return ';' == chr || '#' == chr;
00101       }
00102
00103       /**
00104        * true if character is a valid start character.
00105        */
00106       const bool is_start( char_type chr )
00107       {
00108           return '_' == chr || ::isalpha( chr );
00109       }
00110
00111       /**
00112        * true if character is a valid second and following character.
00113        */
00114       const bool is_follow( char_type chr )
00115       {
00116           return '_' == chr
00117               || ::isalnum( chr )
00118               || std::string::npos != m_tokenizer->m_follow.find_first_of( chr );
00119       }
00120
00121       /**
00122        * true if character is a valid start or following character.
00123        */
00124       const bool is_start_or_follow( char_type chr )
00125       {
00126           return is_start( chr ) || is_follow( chr );
00127       }
00128
00129       /**
00130        * advance to next token.
00131        */
00132       class_type& operator++ ()
00133       {
00134          assert( NULL != m_is );
00135
00136          char_type chr = ' ';
00137          m_token.erase();
00138
00139          /*
00140           * skip non-start characters:
00141           */
00142          while ( *m_is && !is_start( chr = m_is->get() ) )
00143          {
00144             // skip line comments:
00145             if ( is_start_comment( chr ) && m_tokenizer->m_skip_comments )
00146             {
00147                while ( *m_is && '\n' != ( chr = m_is->get() ) )
00148                {
00149                   ;
00150                }
00151             }
00152
00153             // count lines:
00154             if ( '\n' == chr )
00155             {
00156                 ++m_line;
00157             };
00158          }
00159          m_is->putback( chr );
00160
00161          /*
00162           * scan token:
00163           */
00164          while( *m_is && is_start_or_follow( chr = m_is->get() ) )
00165          {
00166             m_token.append( 1, chr );
00167          }
00168          m_is->putback( chr );
00169
00170          /*
00171           * transform to lowercase?
00172           */
00173          if ( m_tokenizer->m_lowercase )
00174          {
00175             std::transform
00176             ( m_token.begin(), m_token.end()
00177             , m_token.begin()
00178             , tolower
00179             );
00180          }
00181
00182          /*
00183           * signal end of input:
00184           */
00185          if ( ! *m_is )
00186          {
00187              m_is = NULL;
00188          }
00189
00190          return *this;
00191       }
00192
00193 //      /**
00194 //       * return current token string.
00195 //       */
00196 //       operator std::string() const
00197 //      {
00198 //         return m_token;
00199 //      }
00200 //
00201 //      /**
00202 //       * return current token string.
00203 //       */
00204 //      operator line_number_type() const
00205 //      {
00206 //         return m_line;
00207 //      }
00208
00209       /**
00210        * return current token string, line number pair.
00211        */
00212       const value_type operator*() const
00213       {
00214          return value_type( m_token, m_line );
00215       }
00216
00217       /**
00218        * true if this and other iterators are unequal.
00219        */
00220       const bool operator!= ( class_type const& rhs ) const
00221       {
00222           return m_is != rhs.m_is;
00223       }
00224
00225    private:
00226       /**
00227        * the input stream; NULL if end-of-input has been reached.
00228        */
00229       std::istream* m_is;
00230
00231       /**
00232        * skip comments flag.
00233        */
00234       basic_tokenizer  const* m_tokenizer;
00235
00236       /**
00237        * current token string.
00238        */
00239       std::string m_token;
00240
00241       /**
00242        * current line number.
00243        */
00244       line_number_type m_line;
00245    };
00246
00247    /**
00248     * constructor.
00249     */
00250    basic_tokenizer( std::istream& is )
00251    : m_is( is )
00252    , m_skip_comments( false )
00253    {
00254       ;
00255    }
00256
00257    /**
00258     * return collection's begin iterator.
00259     */
00260    iterator begin()
00261    {
00262       return iterator( *this );
00263    }
00264
00265    /**
00266     * return collection's end iterator.
00267     */
00268    iterator end()
00269    {
00270       return iterator();
00271    }
00272
00273    /**
00274     * set or clear skip-comments flag.
00275     */
00276    void set_skip_comments( const bool skip = true )
00277    {
00278       m_skip_comments = skip;
00279    }
00280
00281    /**
00282     * add given set of characters to the the start symbol set.
00283     */
00284    void add_to_start_set( std::string const& set )
00285    {
00286       m_start += set;
00287    }
00288
00289    /**
00290     * add given set of characters to the the follow symbol set.
00291     */
00292    void add_to_followset( std::string const& set )
00293    {
00294       m_follow += set;
00295    }
00296
00297    /**
00298     * set or clear the convert-to-lowercase flag.
00299     */
00300    void set_lowercase( const bool lowercase )
00301    {
00302       m_lowercase = lowercase;
00303    }
00304
00305 private:
00306    /**
00307     * the input stream pointer; NULL if end-of-input has been reached.
00308     */
00309    std::istream &m_is;
00310
00311    /**
00312     * skip comments flag.
00313     */
00314    bool m_skip_comments;
00315
00316    /**
00317     * transform tokens to lowercase.
00318     */
00319    bool m_lowercase;
00320
00321    /**
00322     * additional follow set.
00323     */
00324    std::string m_start;
00325
00326    /**
00327     * additional follow set.
00328     */
00329    std::string m_follow;
00330 };
00331
00332 /**
00333  * tokenizer for char character type.
00334  */
00335 typedef basic_tokenizer< char > Tokenizer;
00336
00337 } // namespace wordindex
00338
00339 /*
00340  * end of file
00341  */
00342


WordIndex: Utility.h Source File

Utility.h

Go to the documentation of this file.
00001 /*
00002  * src/utility.h - conversion shims etc.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: Utility.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef _utility_h_included
00030 #define _utility_h_included
00031
00032 #include "Config.h"                     // for configuration
00033
00034 #include <io.h>                         // for _access()
00035 #include <string>                       // for std::string
00036 #include <sstream>                      // for std::stringstream
00037
00038 namespace wordindex {
00039
00040 /**
00041  * base class to implement UnCopyable interface.
00042  */
00043 class UnCopyable
00044 {
00045 public:
00046    /**
00047     * default constructor.
00048     */
00049    UnCopyable() { ; }
00050
00051 private:
00052    /**
00053     * this class type.
00054     */
00055    typedef UnCopyable class_type;
00056
00057    /**
00058     * \name Not To Be Implemented
00059     * @{
00060     */
00061
00062    /// copy-assignment constructor.
00063    UnCopyable( class_type const& );
00064
00065    /// copy-assignment operator.
00066    class_type& operator= ( class_type const& );
00067
00068    /// @}
00069 };
00070
00071 /**
00072  * string to character pointer conversion shim.
00073  */
00074 inline const char *to_charptr( const std::string& s )
00075 {
00076    return s.c_str();
00077 }
00078
00079 /**
00080  * integer to string conversion shim.
00081  */
00082 inline const std::string to_string( long const x )
00083 {
00084    char buffer[32];
00085
00086    return _ltoa( x, buffer, 10 );
00087 }
00088
00089 /**
00090  * integer to string conversion shim.
00091  */
00092 inline const std::string to_string( int const x )
00093 {
00094    return to_string( static_cast< long >( x ) );
00095 }
00096
00097 /**
00098  * integer to string conversion shim.
00099  */
00100 inline const std::string to_string( double const x, int const n = 3 )
00101 {
00102    char buffer[32];
00103
00104    return _gcvt( x, n, buffer );
00105 }
00106
00107 /**
00108  * convert version to string.
00109  */
00110 inline const std::string to_version_string( int const version, int const div )
00111 {
00112    int maj = version / div;
00113    int min = version - maj * div;
00114
00115    return to_string( maj ) + "." + to_string( min );
00116 }
00117
00118 /**
00119  * string to integer conversion shim.
00120  */
00121 inline const long to_int( const char* s )
00122 {
00123    return strtol( s, 0, 0 );
00124 }
00125
00126 /**
00127  * string to real conversion shim.
00128  */
00129 inline const double to_real( const char* s )
00130 {
00131    return strtod( s, 0 );
00132 }
00133
00134 /**
00135  * the filename without the extension.
00136  */
00137 inline const std::string basename( std::string const filename )
00138 {
00139    return filename.substr( 0, filename.find_last_of( "." ) );
00140 }
00141
00142 // TODO (moene#1#): safe?
00143 /**
00144  * the filename without the directory part and extension.
00145  */
00146 inline const std::string filename( std::string const filename )
00147 {
00148    std::string base( basename( filename ) );
00149
00150    int pos = base.find_last_of( "/\\" );
00151
00152    return base.substr( (0 != pos) + pos );
00153 }
00154
00155 /**
00156  * the filename extension.
00157  */
00158 inline const std::string extension( std::string const filename )
00159 {
00160    return filename.substr( filename.find_last_of( "." ) );
00161 }
00162
00163 /**
00164  * true if the specified file is "-" (stdin) or exists as diskfile, false otherwise.
00165  */
00166 inline const bool exist( std::string const filename)
00167 {
00168    return "-" == filename || 0 == _access( to_charptr( filename ), 0 );
00169 }
00170
00171 } // namespace wordindex
00172
00173 #endif //#ifndef _utility_h_included
00174
00175 /*
00176  * end of file
00177  */
00178


WordIndex: WordIndex.h Source File

WordIndex.h

Go to the documentation of this file.
00001 /*
00002  * wordindex.h - word index collection and - insert iterator.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: WordIndex.h 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef wordindex_h_included
00030 #define wordindex_h_included
00031
00032 #include "Pair.h"    // for class wordindex::Pair<>
00033 #include "Utility.h" // for class wordindex::UnCopyable
00034
00035 #include <map>       // for std::map<> (associative array)
00036 #include <vector>    // for std::vector (list if line numbers)
00037 #include <string>    // for std::string
00038
00039 namespace wordindex {
00040
00041 // TODO (Martin#1#): localized sorting?
00042 /**
00043  * std::string comparator for use in map.
00044  */
00045 class StringLess
00046 {
00047 public:
00048   /**
00049    * apply.
00050    */
00051    virtual const bool operator()( std::string const& s1, std::string const& s2 ) const
00052    {
00053       return s1 < s2;
00054    }
00055 };
00056
00057 /**
00058  * std::string comparator for use in map.
00059  */
00060 class NoCaseLess : public StringLess
00061 {
00062 public:
00063   /**
00064    * apply.
00065    */
00066    virtual const bool operator()( std::string const& s1, std::string const& s2 ) const
00067    {
00068       return std::lexicographical_compare
00069       (
00070          s1.begin(), s1.end(),
00071          s2.begin(), s2.end(),
00072          nocase_less
00073       );
00074    }
00075
00076 private:
00077   /**
00078    * character comparator for std::lexicographical_compare().
00079    */
00080    static const bool nocase_less( char const c1, char const c2 )
00081    {
00082        return toupper( c1 ) < toupper( c2 );
00083    }
00084 };
00085
00086 /**
00087  * collect tokens with their associated line numbers.
00088  */
00089 class WordIndex : private UnCopyable
00090 {
00091 private:
00092    /**
00093     * the word type.
00094     */
00095    typedef std::string word_type;
00096
00097    /**
00098     * the line number type.
00099     */
00100    typedef int line_number_type;
00101
00102    /**
00103     * the list of line numbers type.
00104     */
00105    typedef std::vector< line_number_type > locations_type;
00106
00107    /**
00108     * the token--locations associative array (map).
00109     */
00110    typedef std::map< word_type, locations_type, StringLess > map_type;
00111
00112 public:
00113    /**
00114     * this class type.
00115     */
00116    typedef WordIndex class_type;
00117
00118    /**
00119     * this value type.
00120     */
00121    typedef map_type::value_type value_type;
00122
00123    /**
00124     * the token--line number pair type.
00125     */
00126    typedef Pair< word_type, line_number_type > token_type;
00127
00128    /**
00129     * the string--line numbers map iterator type.
00130     */
00131    typedef map_type::iterator iterator;
00132
00133    /**
00134     * the string--line numbers map const iterator type.
00135     */
00136    typedef map_type::const_iterator const_iterator;
00137
00138    /**
00139     * constructor.
00140     */
00141    WordIndex()
00142    : m_lines( 0 )
00143    , m_words( NoCaseLess() )
00144    {
00145       ;
00146    }
00147
00148    /**
00149     * const begin iterator.
00150     */
00151    const_iterator begin() const
00152    {
00153       return const_begin();
00154    }
00155
00156    /**
00157     * const begin iterator.
00158     */
00159    const_iterator const_begin() const
00160    {
00161       return m_words.begin();
00162    }
00163
00164    /**
00165     * const end iterator.
00166     */
00167    const_iterator end() const
00168    {
00169       return const_end();
00170    }
00171
00172    /**
00173     * const end iterator.
00174     */
00175    const_iterator const_end() const
00176    {
00177       return m_words.end();
00178    }
00179
00180    /**
00181     * add a token, line number pair.
00182     */
00183    void insert( token_type const& pair )
00184    {
00185       insert( pair.first, pair.second );
00186    }
00187
00188    /**
00189     * add a token, line number pair.
00190     */
00191    void insert( iterator pos, token_type const& pair )
00192    {
00193       insert( pair.first, pair.second );
00194    }
00195
00196    /**
00197     * add a token and line number.
00198     */
00199    void insert( std::string const s, line_number_type const n )
00200    {
00201       ++m_lines;
00202       m_words[ s ].push_back( n );
00203    }
00204
00205    /**
00206     * number of distinct words.
00207     */
00208    const int words() const
00209    {
00210        return m_words.size();
00211    }
00212
00213    /**
00214     * number of line references.
00215     */
00216    const int lines() const
00217    {
00218       return m_lines;
00219    }
00220
00221 private:
00222    /**
00223     * number of line references.
00224     */
00225    int m_lines;
00226
00227    /**
00228     * datastructure: a map of token--list of linenumbers pairs (associative array).
00229     */
00230    map_type m_words;
00231 };
00232
00233 /**
00234 * insert iterator.
00235 */
00236 #if defined( _WORDINDEX_MSC6 )
00237 template < typename C >
00238 class insert_iterator : public std::iterator< std::output_iterator_tag, void >
00239 #else
00240 template < typename C >
00241 class insert_iterator : public std::iterator< std::output_iterator_tag, void, void, void, void >
00242 #endif
00243 {
00244 public:
00245   /**
00246    * the container type.
00247    */
00248   typedef C container_type;
00249
00250   /**
00251    * this class type.
00252    */
00253   typedef insert_iterator class_type;
00254
00255   /**
00256    * the value type.
00257    */
00258   typedef typename container_type::token_type value_type;
00259
00260   /**
00261    * constructor.
00262    */
00263   explicit insert_iterator( container_type& container )
00264   : m_container( container )
00265   {
00266      ;
00267   }
00268
00269   /**
00270    * pre-increment (no-op).
00271    */
00272   class_type& operator++()
00273   {
00274      return *this;
00275   }
00276
00277   /**
00278    * post-increment (no-op).
00279    */
00280   class_type& operator++( int )
00281   {
00282      return *this;
00283   }
00284
00285   /**
00286    * de-reference.
00287    */
00288   class_type& operator* ()
00289   {
00290      return *this;
00291   }
00292
00293   /**
00294    * insert (add) a value.
00295    */
00296   class_type& operator=( value_type const& value )
00297   {
00298      m_container.insert( value );
00299      return *this;
00300   }
00301
00302 private:
00303   /**
00304    * the container.
00305    */
00306   container_type& m_container;
00307 };
00308
00309 /**
00310  * convenience function to create WordIndex insert_iterator.
00311  */
00312 inline insert_iterator< WordIndex > wordindex_inserter( WordIndex& collection )
00313 {
00314     return insert_iterator< WordIndex >( collection );
00315 }
00316
00317 } // namespace wordindex
00318
00319 #endif // wordindex_h_included
00320
00321 /*
00322  * end of file
00323  */


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::AuthorVisitor Member List

This is the complete list of members for wordindex::AuthorVisitor, including all inherited members.

AuthorVisitor(std::ostream *output, const std::string name) | wordindex::AuthorVisitor | [inline]
m_name | wordindex::AuthorVisitor | [private]
m_output | wordindex::AuthorVisitor | [private]
visit() | wordindex::AuthorVisitor | [inline]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::ClapOutput Member List

This is the complete list of members for wordindex::ClapOutput, including all inherited members.

failure(clp::CmdLineInterface &c, clp::ArgException &e) | wordindex::ClapOutput | [inline, virtual]
usage(clp::CmdLineInterface &c) | wordindex::ClapOutput | [inline, virtual]
version(clp::CmdLineInterface &c) | wordindex::ClapOutput | [inline, virtual]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::FilenameConstraint Member List

This is the complete list of members for wordindex::FilenameConstraint, including all inherited members.

check(const std::string &name) const | wordindex::FilenameConstraint | [inline, virtual]
description() const | wordindex::FilenameConstraint | [inline, virtual]
FilenameConstraint(Logger &logger) | wordindex::FilenameConstraint | [inline]
m_logger | wordindex::FilenameConstraint | [private]
shortID() const | wordindex::FilenameConstraint | [inline, virtual]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Logger Member List

This is the complete list of members for wordindex::Logger, including all inherited members.

Fatal(std::string const &msg) | wordindex::Logger | [inline]
GetLevel() const | wordindex::Logger | [inline]
Logger(const std::string &progname, std::ostream &os=std::cout) | wordindex::Logger | [inline]
m_level | wordindex::Logger | [private]
m_os | wordindex::Logger | [private]
m_progname | wordindex::Logger | [private]
Report(int const level, std::string const &msg) | wordindex::Logger | [inline]
Report(int const level, std::string const &filename, std::string const &msg) | wordindex::Logger | [inline]
ReportPart(int const level, std::string const &msg) | wordindex::Logger | [inline]
SetLevel(int const level) | wordindex::Logger | [inline]
Warning(std::string const &msg) | wordindex::Logger | [inline]
~Logger() | wordindex::Logger | [inline]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::NoCaseLess Member List

This is the complete list of members for wordindex::NoCaseLess, including all inherited members.

nocase_less(char const c1, char const c2) | wordindex::NoCaseLess | [inline, private, static]
operator()(std::string const &s1, std::string const &s2) const | wordindex::NoCaseLess | [inline, virtual]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Printer Member List

This is the complete list of members for wordindex::Printer, including all inherited members.

m_context | wordindex::Printer | [private]
m_options | wordindex::Printer | [private]
m_os | wordindex::Printer | [private]
operator()(value_type const &value) const | wordindex::Printer | [inline]
Printer(std::ostream &os, Options const &options, Context const &context) | wordindex::Printer | [inline]
value_type typedef | wordindex::Printer | [private]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Reader Member List

This is the complete list of members for wordindex::Reader, including all inherited members.

m_context | wordindex::Reader | [private]
m_options | wordindex::Reader | [private]
operator()(value_type const &element) | wordindex::Reader | [inline]
Reader(Options const &options, Context &context) | wordindex::Reader | [inline]
value_type typedef | wordindex::Reader |


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::StringLess Member List

This is the complete list of members for wordindex::StringLess, including all inherited members.

operator()(std::string const &s1, std::string const &s2) const wordindex::StringLess [inline, virtual]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::UnCopyable Member List

This is the complete list of members for wordindex::UnCopyable, including all inherited members.

class_type typedef | wordindex::UnCopyable | [private]
operator=(class_type const &) | wordindex::UnCopyable | [private]
UnCopyable() | wordindex::UnCopyable | [inline]
UnCopyable(class_type const &) | wordindex::UnCopyable | [private]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::WordIndex Member List

This is the complete list of members for wordindex::WordIndex, including all inherited members.

begin() const | wordindex::WordIndex | [inline]
class_type typedef | wordindex::WordIndex |
const_begin() const | wordindex::WordIndex | [inline]
const_end() const | wordindex::WordIndex | [inline]
const_iterator typedef | wordindex::WordIndex |
end() const | wordindex::WordIndex | [inline]
insert(token_type const &pair) | wordindex::WordIndex | [inline]
insert(iterator pos, token_type const &pair) | wordindex::WordIndex | [inline]
insert(std::string const s, line_number_type const n) | wordindex::WordIndex | [inline]
iterator typedef | wordindex::WordIndex |
line_number_type typedef | wordindex::WordIndex | [private]
lines() const | wordindex::WordIndex | [inline]
locations_type typedef | wordindex::WordIndex | [private]
m_lines | wordindex::WordIndex | [private]
m_words | wordindex::WordIndex | [private]
map_type typedef | wordindex::WordIndex | [private]
token_type typedef | wordindex::WordIndex |
UnCopyable() | wordindex::UnCopyable | [inline, private]
value_type typedef | wordindex::WordIndex |
word_type typedef | wordindex::WordIndex | [private]
WordIndex() | wordindex::WordIndex | [inline]
words() const | wordindex::WordIndex | [inline]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::basic_tokenizer< C > Member List

This is the complete list of members for wordindex::basic_tokenizer< C >, including all inherited members.

add_to_followset(std::string const &set) | wordindex::basic_tokenizer< C > | [inline]
add_to_start_set(std::string const &set) | wordindex::basic_tokenizer< C > | [inline]
basic_tokenizer(std::istream &is) | wordindex::basic_tokenizer< C > | [inline]
begin() | wordindex::basic_tokenizer< C > | [inline]
char_type typedef | wordindex::basic_tokenizer< C > |
class_type typedef | wordindex::basic_tokenizer< C > |
end() | wordindex::basic_tokenizer< C > | [inline]
line_number_type typedef | wordindex::basic_tokenizer< C > |
m_follow | wordindex::basic_tokenizer< C > | [private]
m_is | wordindex::basic_tokenizer< C > | [private]
m_lowercase | wordindex::basic_tokenizer< C > | [private]
m_skip_comments | wordindex::basic_tokenizer< C > | [private]
m_start | wordindex::basic_tokenizer< C > | [private]
set_lowercase(const bool lowercase) | wordindex::basic_tokenizer< C > | [inline]
set_skip_comments(const bool skip=true) | wordindex::basic_tokenizer< C > | [inline]
UnCopyable() | wordindex::UnCopyable | [inline, private]
value_type typedef | wordindex::basic_tokenizer< C > |


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::basic_tokenizer< C >::iterator Member List

This is the complete list of members for wordindex::basic_tokenizer< C >::iterator, including all inherited members.

class_type typedef | wordindex::basic_tokenizer< C >::iterator |
is_follow(char_type chr) | wordindex::basic_tokenizer< C >::iterator | [inline]
is_start(char_type chr) | wordindex::basic_tokenizer< C >::iterator | [inline]
is_start_comment(char_type chr) | wordindex::basic_tokenizer< C >::iterator | [inline]
is_start_or_follow(char_type chr) | wordindex::basic_tokenizer< C >::iterator | [inline]
iterator(basic_tokenizer &tokenizer) | wordindex::basic_tokenizer< C >::iterator | [inline, explicit]
iterator() | wordindex::basic_tokenizer< C >::iterator | [inline]
m_is | wordindex::basic_tokenizer< C >::iterator | [private]
m_line | wordindex::basic_tokenizer< C >::iterator | [private]
m_token | wordindex::basic_tokenizer< C >::iterator | [private]
m_tokenizer | wordindex::basic_tokenizer< C >::iterator | [private]
operator *() const | wordindex::basic_tokenizer< C >::iterator | [inline]
operator!=(class_type const &rhs) const | wordindex::basic_tokenizer< C >::iterator | [inline]
operator++() | wordindex::basic_tokenizer< C >::iterator | [inline]


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::contained_in< C > Member List

This is the complete list of members for wordindex::contained_in< C >, including all inherited members.

contained_in(container_type const &c) | wordindex::contained_in< C > | [inline]
container_type typedef | wordindex::contained_in< C > |
m_container | wordindex::contained_in< C > | [private]
operator()(value_type const &value) const | wordindex::contained_in< C > | [inline]
value_type typedef | wordindex::contained_in< C > |


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::insert_iterator< C > Member List

This is the complete list of members for wordindex::insert_iterator< C >, including all inherited members.

class_type typedef | wordindex::insert_iterator< C > |
container_type typedef | wordindex::insert_iterator< C > |
insert_iterator(container_type &container) | wordindex::insert_iterator< C > | [inline, explicit]
m_container | wordindex::insert_iterator< C > | [private]
operator *() | wordindex::insert_iterator< C > | [inline]
operator++() | wordindex::insert_iterator< C > | [inline]
operator++(int) | wordindex::insert_iterator< C > | [inline]
operator=(value_type const &value) | wordindex::insert_iterator< C > | [inline]
value_type typedef | wordindex::insert_iterator< C > |


WordIndex: Class Members - Functions

  • Class List
  • Class Hierarchy
  • Class Members
  • All
  • Functions
  • Variables
  • Typedefs
  • a
  • b
  • c
  • d
  • e
  • f
  • g
  • i
  • l
  • n
  • o
  • p
  • r
  • s
  • u
  • v
  • w
  • ~

- a -

  • add_to_followset() : wordindex::basic_tokenizer< C >
  • add_to_start_set() : wordindex::basic_tokenizer< C >
  • AuthorVisitor() : wordindex::AuthorVisitor

- b -

  • basic_tokenizer() : wordindex::basic_tokenizer< C >
  • begin()
: wordindex::WordIndex
, wordindex::basic_tokenizer< C >

- c -

  • check() : wordindex::FilenameConstraint
  • const_begin() : wordindex::WordIndex
  • const_end() : wordindex::WordIndex
  • contained_in() : wordindex::contained_in< C >

- d -

  • description() : wordindex::FilenameConstraint

- e -

  • end()
: wordindex::basic_tokenizer< C >
, wordindex::WordIndex

- f -

  • failure() : wordindex::ClapOutput
  • Fatal() : wordindex::Logger
  • FilenameConstraint() : wordindex::FilenameConstraint

- g -

  • GetLevel() : wordindex::Logger

- i -

  • insert() : wordindex::WordIndex
  • insert_iterator() : wordindex::insert_iterator< C >
  • is_follow() : wordindex::basic_tokenizer< C >::iterator
  • is_start() : wordindex::basic_tokenizer< C >::iterator
  • is_start_comment() : wordindex::basic_tokenizer< C >::iterator
  • is_start_or_follow() : wordindex::basic_tokenizer< C >::iterator
  • iterator() : wordindex::basic_tokenizer< C >::iterator

- l -

  • lines() : wordindex::WordIndex
  • Logger() : wordindex::Logger

- n -

  • nocase_less() : wordindex::NoCaseLess

- o -

  • operator *() : wordindex::basic_tokenizer< C >::iterator , wordindex::insert_iterator< C >
  • operator const U() : wordindex::Pair< U, V >
  • operator const V() : wordindex::Pair< U, V >
  • operator!=() : wordindex::basic_tokenizer< C >::iterator
  • operator()()
: wordindex::Printer
, wordindex::StringLess
, wordindex::NoCaseLess
, wordindex::contained_in< C >
, wordindex::Reader
  • operator++()
: wordindex::insert_iterator< C >
, wordindex::basic_tokenizer< C >::iterator
, wordindex::insert_iterator< C >
  • operator=()
: wordindex::UnCopyable
, wordindex::Pair< U, V >
, wordindex::insert_iterator< C >
  • Options() : wordindex::Options

- p -

  • Pair() : wordindex::Pair< U, V >
  • Printer() : wordindex::Printer

- r -

  • Reader() : wordindex::Reader
  • Report() : wordindex::Logger
  • ReportPart() : wordindex::Logger

- s -

  • set_lowercase() : wordindex::basic_tokenizer< C >
  • set_skip_comments() : wordindex::basic_tokenizer< C >
  • SetLevel() : wordindex::Logger
  • shortID() : wordindex::FilenameConstraint

- u -

  • UnCopyable() : wordindex::UnCopyable
  • usage() : wordindex::ClapOutput

- v -

  • version() : wordindex::ClapOutput
  • visit() : wordindex::AuthorVisitor

- w -

  • Warning() : wordindex::Logger
  • WordIndex() : wordindex::WordIndex
  • words() : wordindex::WordIndex

- ~ -

  • ~Logger() : wordindex::Logger


WordIndex: Class Members - Typedefs

  • Class List
  • Class Hierarchy
  • Class Members
  • All
  • Functions
  • Variables
  • Typedefs

  • char_type : wordindex::basic_tokenizer< C >
  • class_type
: wordindex::basic_tokenizer< C >
, wordindex::UnCopyable
, wordindex::WordIndex
, wordindex::insert_iterator< C >
, wordindex::basic_tokenizer< C >::iterator
  • const_iterator : wordindex::WordIndex
  • container_type : wordindex::contained_in< C > , wordindex::insert_iterator< C >
  • iterator : wordindex::WordIndex
  • line_number_type : wordindex::basic_tokenizer< C > , wordindex::WordIndex
  • locations_type : wordindex::WordIndex
  • map_type : wordindex::WordIndex
  • token_type : wordindex::WordIndex
  • value_type
: wordindex::contained_in< C >
, wordindex::insert_iterator< C >
, wordindex::WordIndex
, wordindex::basic_tokenizer< C >
, wordindex::Printer
, wordindex::Reader
  • word_type : wordindex::WordIndex


WordIndex: Class Members - Variables

  • Class List
  • Class Hierarchy
  • Class Members
  • All
  • Functions
  • Variables
  • Typedefs
  • f
  • i
  • k
  • l
  • m
  • n
  • r
  • s
  • w

- f -

  • first : wordindex::Pair< U, V >
  • frequency : wordindex::Options

- i -

  • ignorecase : wordindex::Options

- k -

  • keywords : wordindex::Context

- l -

  • lowercase : wordindex::Options

- m -

  • m_container : wordindex::contained_in< C > , wordindex::insert_iterator< C >
  • m_context : wordindex::Printer , wordindex::Reader
  • m_follow : wordindex::basic_tokenizer< C >
  • m_is : wordindex::basic_tokenizer< C >::iterator , wordindex::basic_tokenizer< C >
  • m_level : wordindex::Logger
  • m_line : wordindex::basic_tokenizer< C >::iterator
  • m_lines : wordindex::WordIndex
  • m_logger : wordindex::FilenameConstraint
  • m_lowercase : wordindex::basic_tokenizer< C >
  • m_name : wordindex::AuthorVisitor
  • m_options : wordindex::Printer , wordindex::Reader
  • m_os : wordindex::Logger , wordindex::Printer
  • m_output : wordindex::AuthorVisitor
  • m_progname : wordindex::Logger
  • m_skip_comments : wordindex::basic_tokenizer< C >
  • m_start : wordindex::basic_tokenizer< C >
  • m_token : wordindex::basic_tokenizer< C >::iterator
  • m_tokenizer : wordindex::basic_tokenizer< C >::iterator
  • m_words : wordindex::WordIndex

- n -

  • name_width : wordindex::Options

- r -

  • reverse : wordindex::Options

- s -

  • second : wordindex::Pair< U, V >
  • summary : wordindex::Options

- w -

  • wordindex : wordindex::Context


WordIndex: Class Members

  • File List
  • File Members
  • All
  • Functions
  • Defines

  • dimensionof : main.cpp
  • typename_type_k : Config.h
  • WORDINDEX_RELEASE_DATE_STRING : version.h
  • WORDINDEX_RELEASE_NUMBER_STRING : version.h


WordIndex: Class Members

  • File List
  • File Members
  • All
  • Functions
  • Defines

  • main() : main.cpp


WordIndex: main.cpp Source File

main.cpp

Go to the documentation of this file.
00001 /*
00002  * main.cpp - program's main line.
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id: main.cpp 10 2008-01-08 21:52:32Z moene $
00023  */
00024
00025 /*
00026  * compile: cl /EHsc -I../include -Fewordindex.exe main.cpp
00027  * compile: g++ -I../include -o wordindex.exe main.cpp
00028  */
00029
00030 /*
00031  *   keywords: 33
00032  *      words: 1
00033  * references: 2
00034  *
00035  *      hello: 100% (2)  5 33
00036  */
00037
00038 /*
00039  * handle options:
00040  * ideas:
00041  * - case sensitivity +case=(preserve,ignore) [preserve]
00042  * - number of lines per page +pagelength=n [1]
00043  * - tokens to accept
00044  * - keyword tokens (stop-words) +stopwords=filename [none]
00045  *
00046  * report:
00047  * - report-summary: keywords, words, references
00048  * - report-xxx:
00049  * - number of distinct keywords (stopwords): +report=count_keywords
00050  * - number of distinct words: +report=count_words
00051  * - number of word occurrences: +report=count_keywords
00052  * - percentage of word occurrences
00053  * - lines with word occurrences: +report=
00054  * - shorthand: +report=count_words,...
00055  */
00056
00057 /**
00058  * \file
00059  */
00060
00061 #include "Config.h"     // for configuration
00062 #include "Logger.h"     // for class Logger
00063 #include "Pair.h"       // for pair_type
00064 #include "Tokenizer.h"  // for class Tokenizer
00065 #include "Utility.h"    // shims
00066 #include "Version.h"    // for WORDINDEX_VERSION_STRING
00067 #include "WordIndex.h"  // for class WordIndex
00068
00069 #include <ctype.h>     // for ::isalpha()
00070
00071 #include <algorithm> // for std::copy()
00072 #include <set>       // for std::ste<> (associative array)
00073 #include <iterator>  // for std::iterator<> base class
00074 #include <iomanip>   // for std::setw() etc.
00075 #include <fstream>   // for std::ifstream
00076 #include <iostream>  // for std::cin, std::cout
00077 #include <string>    // for std::string
00078 //#include <utility>   // for std::pair<>
00079 #include <vector>    // for std::vector (list if line numbers)
00080
00081 #include <tclap/CmdLine.h>    // for templatized C++ command line parser library
00082
00083 /**
00084  * the dimension of an array.
00085  */
00086 #define dimensionof( a ) ( sizeof(a) / sizeof( *(a) ) )
00087
00088 /**
00089  * clp namespace shorthand for TCLAP.
00090  */
00091 namespace clp = TCLAP;
00092
00093 /**
00094  * namespace for this application.
00095  */
00096 namespace wordindex {
00097
00098 /**
00099  * \name Configuration
00100  * @{
00101  */
00102
00103 /**
00104  * the author name.
00105  */
00106 std::string const author_string( "Martin J. Moene <m.j.moene@eld.physics.LeidenUniv.nl>" );
00107
00108 /**
00109  * the program version.
00110  */
00111 std::string const version_string( WORDINDEX_RELEASE_NUMBER_STRING );
00112
00113 /**
00114  * the program date.
00115  */
00116 std::string const date_string   ( WORDINDEX_RELEASE_DATE_STRING );
00117
00118 /**
00119  * the program name / basename.
00120  */
00121 std::string /* const */ program_name( "wordindex" );
00122
00123 /**
00124  * the program banner.
00125  */
00126 std::string const banner(
00127    program_name + "  " + version_string + " (" + date_string + ")  create alphabetically sorted index of words.\n\n"
00128 );
00129
00130 /**
00131  * the program copyright.
00132  */
00133 std::string const copyright(
00134    "Copyright 2007-2008, by Martin J. Moene.\n"
00135 );
00136
00137 /// @} Configuration-End
00138
00139 /**
00140  * hint to read help.
00141  */
00142 std::string const try_help( "Try '" + program_name + " --help' for more information.\n" );
00143
00144 /**
00145  * the line number type.
00146  */
00147 typedef int line_number_type;
00148
00149 /**
00150  * the word type.
00151  */
00152 typedef std::string word_type;
00153
00154 /**
00155  * the filename type.
00156  */
00157 typedef std::string filename_type;
00158
00159 /**
00160  * the keyword container.
00161  */
00162 typedef std::set< word_type > keyword_collection_type;
00163
00164 /**
00165  * the keyword container.
00166  */
00167 typedef keyword_collection_type Keywords;
00168
00169 /**
00170  * the filename list element type.
00171  */
00172 typedef Pair< filename_type, line_number_type > filename_list_element_type;
00173
00174 /**
00175  * the filename list type.
00176  */
00177 typedef std::vector< filename_list_element_type > filename_list_type;
00178
00179 /**
00180  * print collections contents.
00181  */
00182 template < typename C >
00183 void print_collection( std::ostream& os, C const& collection )
00184 {
00185    std::copy
00186    ( collection.begin()
00187    , collection.end()
00188    , std::ostream_iterator< typename_type_k C::value_type >( os, "  " )
00189    );
00190 }
00191
00192 /**
00193  * the logger instance (external linkage).
00194  */
00195 Logger logger( program_name, std::cout );
00196
00197 /**
00198  * the program usage.
00199  */
00200 int usage( std::ostream& os, int status )
00201 {
00202    os <<
00203 //      banner <<
00204       "Usage: " << program_name << " [option...] [file...]\n"
00205       "\n"
00206       "  -h, --help          display this help and exit\n"
00207       "  -a, --author        report authors name and e-mail [no]\n"
00208       "      --version       report program and compiler versions [no]\n"
00209       "  -v, --verbose       report ... [none]\n"
00210       "\n"
00211       "  -f, --frequency     also report word frequency as d.dd% (n) [no]\n"
00212 //      "  -g, --ignorecase    handle upper and lowercase as being equivalent [no]\n"
00213       "  -l, --lowercase     transform words to lowercase [no]\n"
00214       "  -r, --reverse       only collect keyword occurrences, see --keywords [no]\n"
00215       "  -s, --summary       also report number of (key)words and references [no]\n"
00216       "\n"
00217       "  -i, --input=file    read filenames from given file [standard input or given filenames]\n"
00218       "  -o, --output=file   write output to given file [standard output]\n"
00219       "  -k, --keywords=file read keywords to skip (stopwords) from given file [none]\n"
00220       "\n"
00221       "Long options also may start with a plus, like: +help.\n"
00222       "\n" <<
00223       filename( program_name )  << " creates an alphabetically sorted index of words present in the\n"
00224       "input files and it reports the lines where those words occur.\n"
00225       "Words that are marked as keywords are excluded (see option --keywords).\n"
00226       "Use option --reverse to only show the occurrences of keywords.\n"
00227       "\n"
00228       "Words can be read from standard input, or from files specified on the command\n"
00229       "line and from files that are specified in another file (see option --input).\n"
00230       "\n"
00231       "A file that specifies input filenames may look as follows:\n"
00232       "   # comment that extends to the end of the line ( ; also starts comment line)\n"
00233       "   file1.txt file2.txt\n"
00234       "   file3.txt\n"
00235       "\n"
00236       "Example:\n"
00237       "   echo hello world | " << filename( program_name ) << " +summary +frequency\n"
00238       "       keywords  0\n"
00239       "          words  2\n"
00240       "     references  2\n"
00241       "\n"
00242       "          hello  50% (1)  1\n"
00243       "          world  50% (1)  1\n"
00244       "\n"
00245       "Example:\n"
00246       "   " << filename( program_name )  << " +lowercase file.txt | sort -n -k2 -r \n"
00247       "This creates a list of lowercase words, sorted on frequency of occurrence.\n"
00248       "\n"
00249 //      << copyright
00250       ;
00251
00252    return status;
00253 }
00254
00255 /**
00256  * the version help.
00257  */
00258 int version( std::ostream& os, int status )
00259 {
00260    os <<
00261       banner <<
00262       "Compiled with "
00263 #if defined( __GNUC__ )
00264       "GNU C++ version "  << __GNUC__ << "." << __GNUC_MINOR__ <<
00265 // intel must precede MS VC++, because it maskerades as MS VC++
00266 #elif defined( __INTEL_COMPILER )
00267       "Intel C++ version " << to_version_string( __INTEL_COMPILER, 100 ) <<
00268 #elif defined( _MSC_VER )
00269       "Microsoft Visual C++ version " << to_version_string( _MSC_VER - 600, 100 ) << " (" << _MSC_VER << ")"
00270 #endif
00271       " on " << __DATE__ <<  ".\n\n" <<
00272       copyright <<
00273       "\nThis is free software; see the source for copying conditions.  There is NO"
00274       "\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." <<
00275       std::endl;
00276
00277    return status;
00278 }
00279
00280 // TODO (Martin#1#): expand, document
00281 /**
00282  * context for read and print.
00283  */
00284 struct Context
00285 {
00286    Keywords keywords;      ///< the keywords specified
00287    WordIndex wordindex;    ///< the non-keywords collected
00288 };
00289
00290 // TODO (Martin#1#): expand, document
00291 /**
00292  * program's options.
00293  */
00294 struct Options
00295 {
00296 public:
00297    /**
00298     * constructor.
00299     */
00300    Options()
00301    : frequency ( false )
00302    , ignorecase( false )
00303    , lowercase ( false )
00304    , reverse   ( false )
00305    , summary   ( false )
00306    , name_width( 20 )
00307    {
00308    }
00309
00310    bool frequency;   ///< report word usage percentage and count
00311    bool ignorecase;  ///< (currently not used)
00312    bool lowercase;   ///< convert words to lowercase
00313    bool reverse;     ///< only report keyword (stopword) usage
00314    bool summary;     ///< also report number of (key)words and references
00315
00316    int  name_width;  ///< name field width
00317 };
00318
00319 /**
00320  * filename list entry exists predicate.
00321  */
00322 void check_file_exists( filename_list_element_type const& element )
00323 {
00324    filename_type    const filename   = element.first;
00325    line_number_type const linenumber = element.second;
00326
00327    if ( !exist( filename ) )
00328    {
00329       if ( 0 == linenumber )
00330       {
00331          logger.Fatal( "cannot open file '" + filename + "'." );
00332       }
00333       else
00334       {
00335          logger.Fatal( "filename list, line " + to_string( linenumber ) + ": cannot open file '" + filename + "'." );
00336       }
00337    }
00338 }
00339
00340 /**
00341  * value contained in collection predicate.
00342  */
00343 template < typename C >
00344 class contained_in : public std::unary_function< typename C::value_type, bool >
00345 {
00346 public:
00347    /**
00348     * the set type.
00349     */
00350    typedef C container_type;
00351
00352    /**
00353     * the value type.
00354     */
00355    typedef typename container_type::value_type value_type;
00356
00357    /**
00358     * the value type.
00359     */
00360    contained_in( container_type const& c )
00361    : m_container( c )
00362    {
00363       ;
00364    }
00365
00366    /**
00367     * the value type.
00368     */
00369    bool operator()( value_type const& value ) const
00370    {
00371       return m_container.count( value ) > 0;
00372    }
00373
00374 private:
00375    /**
00376     * the set.
00377     */
00378    container_type const& m_container;
00379 };
00380
00381 /**
00382  * read words from the given stream into the wordindex.
00383  */
00384 void read( std::istream& is, Options const& options, Context& context )
00385 {
00386    logger.Report( 1, "read()\n" );
00387
00388    /*
00389     * read input tokens, add token-linenumber pairs to wordindex:
00390     */
00391    Tokenizer tokenizer( is );
00392    //   tokenizer.lines_per_page( opt_lines_per_page );
00393    tokenizer.set_lowercase( options.lowercase );
00394
00395    if ( options.reverse ) // include only keywords
00396    {
00397       std::remove_copy_if
00398       ( tokenizer.begin(), tokenizer.end()
00399       , wordindex_inserter( context.wordindex )
00400       , std::not1( contained_in< Keywords >( context.keywords ) )
00401       );
00402    }
00403    else // include only non-keywords
00404    {
00405       std::remove_copy_if
00406       ( tokenizer.begin(), tokenizer.end()
00407       , wordindex_inserter( context.wordindex )
00408       , contained_in< Keywords >( context.keywords )
00409       );
00410    }
00411 }
00412
00413 /**
00414  * process a file.
00415  */
00416 class Reader
00417 {
00418 public:
00419    /**
00420     * the value type.
00421     */
00422    typedef filename_list_element_type value_type;
00423
00424    /**
00425     * constructor.
00426     */
00427    Reader( Options const& options, Context& context )
00428    : m_options( options )
00429    , m_context( context )
00430    {
00431    }
00432
00433    /**
00434     * read the given file.
00435     */
00436    void operator()( value_type const& element )
00437    {
00438       filename_type const filename( element.first );
00439
00440       std::ifstream is( to_charptr( filename ) );
00441
00442       if ( !is )
00443       {
00444          logger.Fatal( "cannot open file '" + filename + "'." );
00445       }
00446
00447       read( is, m_options, m_context );
00448    }
00449
00450 private:
00451    /**
00452     * the options.
00453     */
00454    Options const& m_options;
00455
00456    /**
00457     * the context (words and keywords).
00458     */
00459    Context& m_context;
00460 };
00461
00462 /**
00463  * function object to print an entry from the colleced words.
00464  */
00465 class Printer
00466 {
00467    /**
00468     * the value type.
00469     */
00470    typedef WordIndex::value_type value_type;
00471
00472 public:
00473    /**
00474     * constructor.
00475     */
00476    Printer( std::ostream& os, Options const& options, Context const& context )
00477    : m_os( os )
00478    , m_options( options )
00479    , m_context( context )
00480    {
00481       ; // do nothing
00482    }
00483
00484    /**
00485     * print the given entry.
00486     */
00487    void operator()( value_type const& value ) const
00488    {
00489       const int    count = value.second.size();
00490       const double perct = 100.0 * value.second.size() / m_context.wordindex.lines();
00491
00492       m_os <<
00493          std::setw(m_options.name_width) << std::right << value.first << "  ";
00494
00495       if ( m_options.frequency )
00496       {
00497          m_os <<
00498             std::setw(3) << std::setprecision(3) << perct << "%" << " (" << count << ")  ";
00499       }
00500       print_collection( m_os, value.second );
00501
00502       m_os << std::endl;
00503    }
00504
00505 private:
00506    /**
00507     * the output stream.
00508     */
00509    std::ostream& m_os;
00510
00511    /**
00512     * the options.
00513     */
00514
00515    Options const& m_options;
00516
00517    /**
00518     * the context (words and keywords).
00519     */
00520    Context const& m_context;
00521 };
00522
00523 /**
00524  * print the collected words.
00525  */
00526 void print( std::ostream& os, Options const& options, Context const& context )
00527 {
00528    logger.Report( 1, "print()\n" );
00529
00530    /*
00531     * report wordindex contents:
00532     */
00533    if ( options.summary )
00534    {
00535       os <<
00536          std::setw( options.name_width ) <<   "keywords" << "  " << context.keywords.size() << std::endl <<
00537          std::setw( options.name_width ) <<      "words" << "  " << context.wordindex.words() << std::endl <<
00538          std::setw( options.name_width ) << "references" << "  " << context.wordindex.lines() << std::endl << std::endl;
00539    }
00540
00541    std::for_each
00542    ( context.wordindex.begin()
00543    , context.wordindex.end()
00544    , Printer( os, options, context )
00545    );
00546 }
00547
00548 /**
00549  * user defined output for the tclap commandline handling.
00550  */
00551 class ClapOutput : public clp::StdOutput
00552 {
00553 public:
00554    /**
00555     * fatal error, show help hint, terminate program.
00556     */
00557    virtual void failure( clp::CmdLineInterface& c, clp::ArgException& e )
00558    {
00559       logger.Fatal( std::string( e.what() ) +  "\n" + try_help );
00560    }
00561
00562    /**
00563     * show program usage, terminate program.
00564     */
00565    virtual void usage( clp::CmdLineInterface& c )
00566    {
00567       exit( wordindex::usage( std::cout, 0 ) );
00568    }
00569
00570    /**
00571     * show program version, terminate program.
00572     */
00573    virtual void version( clp::CmdLineInterface& c )
00574    {
00575       exit( wordindex::version( std::cout, 0 ) );
00576    }
00577 };
00578
00579 /**
00580  * handle --author option.
00581  */
00582 class AuthorVisitor : public clp::Visitor
00583 {
00584 private:
00585    std::ostream* m_output;       ///< the output stream
00586    std::string   m_name;         ///< the author name
00587 public:
00588    /**
00589     * constructor.
00590     */
00591    AuthorVisitor( std::ostream* output, const std::string name )
00592    : m_output( output )
00593    , m_name  ( name   )
00594    {
00595       ; // do nothing
00596    }
00597
00598    /**
00599     * handle this option (call-back).
00600     */
00601    void visit()
00602    {
00603       (*m_output) << m_name << std::endl; exit( 0 );
00604    }
00605 };
00606
00607 /**
00608  * tclap commandline filename constraint(s).
00609  */
00610 class FilenameConstraint : public clp::Constraint< std::string >
00611 {
00612 private:
00613    /**
00614     * logger to report through.
00615     */
00616    Logger& m_logger;
00617
00618 public:
00619    /**
00620     * constructor.
00621     */
00622    FilenameConstraint( Logger& logger )
00623    : m_logger( logger )
00624    {
00625       ; // do nothing
00626    }
00627
00628    /**
00629     * constraint description.
00630     */
00631    virtual std::string description() const
00632    {
00633       return "no non-option arguments expected";
00634    }
00635
00636    /**
00637     * constraint id.
00638     */
00639    virtual std::string shortID() const
00640    {
00641       return "filename constraint";
00642    }
00643
00644    /**
00645     * true if filename does not start with - or +.
00646     */
00647    virtual bool check( const std::string& name ) const
00648    {
00649       // string-as-container approach (Kevlin Henney:  The Next Best String)
00650
00651       const std::string set = "+-";
00652
00653       if ( name.begin() != std::find_first_of( name.begin(), name.end(), set.begin(), set.end() ) )
00654 //      if ( std::string::npos == value.find_first_of( "-+" ) )
00655       {
00656          return true;
00657       }
00658
00659       logger.Fatal
00660          ( "unrecognized option '" + name + "' found; review the commandline.\n"
00661          + "Try '" + program_name + " --help' for more information.\n"
00662       );
00663       return false;
00664    }
00665 };
00666
00667 /**
00668  * convert a filename to a filename list element.
00669  */
00670 const filename_list_element_type
00671 to_filename_list_element( filename_type const& filename )
00672 {
00673    return filename_list_element_type( filename );
00674 }
00675
00676 /**
00677  * the program.
00678  */
00679 int main( int argc, char* argv[] )
00680 {
00681    program_name = argv[0];
00682
00683    /*
00684     * handle commandline processing exceptions:
00685     */
00686    try
00687    {
00688       /*
00689        * convenience using declarations and typedefs:
00690        */
00691       using   clp::SwitchArg;
00692       using   clp::MultiSwitchArg;
00693       typedef clp::ValueArg< int         > IntArg;
00694       typedef clp::ValueArg< double      > RealArg;
00695       typedef clp::ValueArg< std::string > StringArg;
00696       typedef clp::UnlabeledMultiArg< filename_type > FileArgs;
00697
00698       /*
00699        * define the commanline parser and options:
00700        */
00701       clp::CmdLine cmd( "unused-message", '=', version_string );
00702       cmd.setOutput( new ClapOutput );
00703
00704 // TODO (Martin#1#): false => not-required
00705
00706            SwitchArg clpAuthor    ( "a", "author"         , "", cmd, false, new AuthorVisitor( &std::cout, author_string ) );
00707       MultiSwitchArg clpVerbose   ( "v", "verbose"        , "", cmd, false );
00708
00709            SwitchArg clpFrequency ( "f", "frequency"      , "", cmd, false );
00710 //           SwitchArg clpIgnorecase( "g", "ignorecase"     , "", cmd, false );
00711            SwitchArg clpLowercase ( "l", "lowercase"      , "", cmd, false );
00712            SwitchArg clpReverse   ( "r", "reverse"        , "", cmd, false );
00713            SwitchArg clpSummary   ( "s", "summary"        , "", cmd, false );
00714
00715            StringArg clpInput     ( "i", "input"          , "file with filenames", false, "[none]", "filename", cmd );
00716            StringArg clpOutput    ( "o", "output"         , "outut file", false, "standard output", "filename", cmd );
00717            StringArg clpKeywords  ( "k", "keywords"       , "keyword file", false, "[none]", "filename", cmd );
00718
00719 //            FileArgs fileArgs    (  "", "filenames"      , false, "type-descr.", cmd, false );
00720             FileArgs fileArgs    (  "", "filenames"      , false, new FilenameConstraint( logger ), cmd );
00721
00722       /*
00723        * parse the command line:
00724        */
00725       cmd.parse( argc, argv );
00726
00727       Options options;
00728       Context context;
00729
00730       /*
00731        * default output file (std::cout):
00732        * FIXME: pointer?
00733        */
00734       std::ostream* output = &std::cout;
00735
00736       /*
00737        * set verbosity:
00738        */
00739       logger.SetLevel( logger.GetLevel() + clpVerbose.getValue() );
00740
00741       /*
00742        * flags:
00743        */
00744       options.frequency  = clpFrequency.isSet();
00745 //      options.ignorecase = clpIgnorecase.isSet();
00746       options.lowercase  = clpLowercase.isSet();
00747       options.reverse    = clpReverse.isSet();
00748       options.summary    = clpSummary.isSet();
00749
00750 //      if ( options.ignorecase )
00751 //      {
00752 //         logger.Fatal( "option --ignorecase is not yet implemented." );
00753 //      }
00754
00755       /*
00756        * open output file if requested:
00757        */
00758       if ( clpOutput.isSet() )
00759       {
00760          const filename_type filename( clpKeywords.getValue() );
00761
00762          output = new std::ofstream( to_charptr( filename ) );
00763
00764          if ( !*output )
00765          {
00766             logger.Fatal( "cannot open file '" + filename + "' for output." );
00767          }
00768       }
00769
00770       /*
00771        * read keywords if requested:
00772        */
00773       if ( clpKeywords.isSet() )
00774       {
00775          const filename_type filename( clpKeywords.getValue() );
00776
00777          std::ifstream is( to_charptr( filename ) );
00778
00779          if ( !is )
00780          {
00781             logger.Fatal( "cannot open file '" + filename + "' for output." );
00782          }
00783          else // FIXME: => read( is, options, context, ctx.keywords ); ??
00784          {
00785             Tokenizer tokenizer( is );
00786
00787             std::copy
00788             ( tokenizer.begin(), tokenizer.end()
00789             , std::inserter( context.keywords, context.keywords.begin() )
00790             );
00791          }
00792       }
00793
00794       /*
00795        * print banner:
00796        */
00797       logger.Report( 1, banner );
00798
00799       filename_list_type filename_list;
00800
00801       /*
00802        * copy filenames from commandline into filename list:
00803        */
00804       std::transform
00805       ( fileArgs.getValue().begin(), fileArgs.getValue().end()
00806       , std::back_inserter( filename_list )
00807       , to_filename_list_element
00808       );
00809
00810       /*
00811        * read filenames from file if requested:
00812        */
00813       if ( clpInput.isSet() )
00814       {
00815          filename_type filename = clpInput.getValue();
00816
00817          if ( !exist( filename ) )
00818          {
00819             logger.Fatal( "cannot open file '" + filename + "' to read filenames from." );
00820          }
00821
00822          std::ifstream is( to_charptr( filename ) );
00823
00824 // TODO (Martin#1#): somehow combine with a read() => lowercase auto synchronizes word-keyword
00825          Tokenizer tokenizer( is );
00826          tokenizer.set_lowercase( options.lowercase );
00827          // FIXME: valid filename characters?
00828 //         tokenizer.filename_scanner();
00829          tokenizer.add_to_followset( "-." );
00830          tokenizer.set_skip_comments();
00831
00832          std::copy
00833          ( tokenizer.begin(), tokenizer.end()
00834          , std::back_inserter( filename_list )
00835          );
00836       }
00837
00838       /*
00839        * check files for existence:
00840        */
00841       std::for_each( filename_list.begin(), filename_list.end(), check_file_exists );
00842
00843       /*
00844        * process given files, or std::cin if none given:
00845        */
00846       if ( filename_list.size() <= 0 )
00847       {
00848          read( std::cin, options, context );
00849       }
00850       else
00851       {
00852          std::for_each
00853          ( filename_list.begin(), filename_list.end()
00854          , Reader( options, context )
00855          );
00856       }
00857
00858       print( *output, options, context );
00859
00860 // TODO (Martin#1#): smart-pointer?, leaking?
00861       if ( output != &std::cout )
00862       {
00863          delete output;
00864       }
00865    }
00866    catch ( clp::ArgException& e )
00867    {
00868       logger.Fatal( e.error() + " " + e.argId() );
00869    }
00870
00871    // FIXME: scoped; reopen()
00872    return 0;
00873 }
00874
00875 } // namespace wordindex
00876
00877 /**
00878  * the program.
00879  */
00880 int main( int argc, char* argv[] )
00881 {
00882    return wordindex::main( argc, argv );
00883 }
00884
00885 /*
00886  * end of file
00887  */


WordIndex: Class Members

  • Namespace List
  • Namespace Members
  • All
  • Functions
  • Variables
  • Typedefs

  • author_string() : wordindex
  • banner() : wordindex
  • basename() : wordindex
  • check_file_exists() : wordindex
  • copyright() : wordindex
  • date_string() : wordindex
  • exist() : wordindex
  • extension() : wordindex
  • filename() : wordindex
  • main() : wordindex
  • print() : wordindex
  • print_collection() : wordindex
  • program_name() : wordindex
  • read() : wordindex
  • to_charptr() : wordindex
  • to_filename_list_element() : wordindex
  • to_int() : wordindex
  • to_real() : wordindex
  • to_string() : wordindex
  • to_version_string() : wordindex
  • try_help() : wordindex
  • usage() : wordindex
  • version() : wordindex
  • version_string() : wordindex
  • wordindex_inserter() : wordindex


WordIndex: Class Members

  • Namespace List
  • Namespace Members
  • All
  • Functions
  • Variables
  • Typedefs

  • filename_list_element_type : wordindex
  • filename_list_type : wordindex
  • filename_type : wordindex
  • keyword_collection_type : wordindex
  • Keywords : wordindex
  • line_number_type : wordindex
  • Tokenizer : wordindex
  • word_type : wordindex


WordIndex: Class Members

  • Namespace List
  • Namespace Members
  • All
  • Functions
  • Variables
  • Typedefs

  • logger : wordindex


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Context Member List

This is the complete list of members for wordindex::Context, including all inherited members.

keywords | wordindex::Context
wordindex | wordindex::Context


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Options Member List

This is the complete list of members for wordindex::Options, including all inherited members.

frequency | wordindex::Options |
ignorecase | wordindex::Options |
lowercase | wordindex::Options |
name_width | wordindex::Options |
Options() | wordindex::Options | [inline]
reverse | wordindex::Options |
summary | wordindex::Options |


WordIndex: Member List

  • Class List
  • Class Hierarchy
  • Class Members

wordindex::Pair< U, V > Member List

This is the complete list of members for wordindex::Pair< U, V >, including all inherited members.

first | wordindex::Pair< U, V > |
operator const U() const | wordindex::Pair< U, V > | [inline]
operator const V() const | wordindex::Pair< U, V > | [inline]
operator=(U const &u) | wordindex::Pair< U, V > | [inline]
operator=(V const &v) | wordindex::Pair< U, V > | [inline]
Pair() | wordindex::Pair< U, V > | [inline]
Pair(U const &u, V const &v) | wordindex::Pair< U, V > | [inline]
Pair(U const &u) | wordindex::Pair< U, V > | [inline, explicit]
Pair(V const &v) | wordindex::Pair< U, V > | [inline, explicit]
second | wordindex::Pair< U, V > |


WordIndex: version.h Source File

version.h

Go to the documentation of this file.
00001 /*
00002  * version.h - DO NOT EDIT, file generated from version.h_in .
00003  *
00004  * This file is part of WordIndex.
00005  *
00006  * Copyright (C) 2007-2008, Martin J. Moene.
00007  *
00008  * This program is free software; you can redistribute it and/or modify
00009  * it under the terms of the GNU General Public License as published by
00010  * the Free Software Foundation; either version 2 of the License, or
00011  * (at your option) any later version.
00012  *
00013  * The program is distributed in the hope that it will be useful,
00014  * but WITHOUT ANY WARRANTY; without even the implied warranty of
00015  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
00016  * GNU General Public License for more details.
00017  *
00018  * You should have received a copy of the GNU General Public License
00019  * along with mngdriver; if not, write to the Free Software
00020  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
00021  *
00022  * $Id:$
00023  */
00024
00025 /**
00026  * \file
00027  */
00028
00029 #ifndef version_h_in_included
00030 #define version_h_in_included
00031
00032 #define WORDINDEX_RELEASE_NUMBER_STRING "0.0.2"        ///< version number as string
00033 #define WORDINDEX_RELEASE_DATE_STRING   "08 Feb 2008"   ///< release date as string
00034
00035 #endif // version_h_in_included
00036
00037 /*
00038  * end of file
00039  */