kyotocabinet::MapReduce Class Reference

MapReduce framework. More...

#include <kcdbext.h>

List of all members.

Classes
class	MapChecker
	Checker for the map process.
class	MapEmitter
	Data emitter for the mapper. More...
class	MapVisitor
	Visitor for the map process.
struct	MergeLine
	Front line of a merging list.
class	ValueIterator
	Value iterator for the reducer. More...
Public Types
enum	Option { XNOLOCK = 1 << 0, XNOCOMP = 1 << 1 }
	Execution options. More...
Public Member Functions
	MapReduce ()
	Default constructor.
virtual	~MapReduce ()
	Destructor.
virtual bool	map (const char *kbuf, size_t ksiz, const char *vbuf, size_t vsiz, MapEmitter *emitter)=0
	Map a record data.
virtual bool	reduce (const char *kbuf, size_t ksiz, ValueIterator *iter)=0
	Reduce a record data.
virtual bool	log (const char *name, const char *message)
	Process a log message.
bool	execute (BasicDB *db, const std::string &tmppath="", uint32_t opts=0)
	Execute the MapReduce process about a database.
void	tune_storage (int32_t dbnum, int64_t clim, int64_t cbnum)
	Set the storage configurations.

---

Detailed Description

MapReduce framework.

Note:

Although this framework is not distributed or concurrent, it is useful for aggregate calculation with less CPU loading and less memory usage.

---

Member Enumeration Documentation

enum kyotocabinet::MapReduce::Option

Execution options.

Enumerator:

XNOLOCK	avoid locking against update operations
XNOCOMP	avoid compression of temporary databases

---

Constructor & Destructor Documentation

kyotocabinet::MapReduce::MapReduce

(

)

[explicit]

Default constructor.

virtual kyotocabinet::MapReduce::~MapReduce

(

)

[virtual]

Destructor.

---

Member Function Documentation

virtual bool kyotocabinet::MapReduce::map	(	const char *	kbuf,
		size_t	ksiz,
		const char *	vbuf,
		size_t	vsiz,
		MapEmitter *	emitter
	)			[pure virtual]

Map a record data.

Parameters:

	kbuf	the pointer to the key region.
	ksiz	the size of the key region.
	vbuf	the pointer to the value region.
	vsiz	the size of the value region.
	emitter	the emitter object.

Returns:

true on success, or false on failure.

Note:

To avoid deadlock, any database operation must not be performed in this function.

virtual bool kyotocabinet::MapReduce::reduce	(	const char *	kbuf,
		size_t	ksiz,
		ValueIterator *	iter
	)			[pure virtual]

Reduce a record data.

Parameters:

	kbuf	the pointer to the key region.
	ksiz	the size of the key region.
	iter	the iterator to get the values.

Returns:

true on success, or false on failure.

Note:

Database operations can be performed in this function.

virtual bool kyotocabinet::MapReduce::log	(	const char *	name,
		const char *	message
	)			[virtual]

Process a log message.

Parameters:

	name	the name of the event.
	message	a supplement message.

Returns:

true on success, or false on failure.

bool kyotocabinet::MapReduce::execute	(	BasicDB *	db,
		const std::string &	tmppath = "",
		uint32_t	opts = 0
	)

Execute the MapReduce process about a database.

Parameters:

	db	the source database.
	tmppath	the path of a directory for the temporary data storage. If it is an empty string, temporary data are handled on memory.
	opts	the optional features by bitwise-or: MapReduce::XNOLOCK to avoid locking against update operations by other threads, MapReduce::XNOCOMP to avoid compression of temporary databases.

Returns:

true on success, or false on failure.

void kyotocabinet::MapReduce::tune_storage	(	int32_t	dbnum,
		int64_t	clim,
		int64_t	cbnum
	)

Set the storage configurations.

Parameters:

	dbnum	the number of temporary databases.
	clim	the limit size of the internal cache.
	cbnum	the bucket number of the internal cache.