eu.dnetlib.data.mapreduce.hbase.broker.enrich

Class AbstractEnrichmentReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,ImmutableBytesWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

eu.dnetlib.data.mapreduce.hbase.broker.enrich.AbstractEnrichmentReducer

Direct Known Subclasses:

EnrichmentReducer, ProjectEnrichmentReducer

public abstract class AbstractEnrichmentReducer
extends org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,ImmutableBytesWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Created by claudio on 20/02/2017.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field and Description
protected eu.dnetlib.pace.config.DedupConfig	dedupConf
protected Set<String>	dsBlacklist
protected Map<String,String>	dsTypeMap
protected Set<String>	dsTypeWhitelist
protected Set<String>	dsWhitelist
protected static int	LIMIT
protected double	scaleLB lower bound for trust scaling
protected static int	SCORE_DECIMALS
protected Set<String>	untrustedOaDsList

Constructor Summary

Constructors

Constructor and Description
AbstractEnrichmentReducer()

Method Summary

Modifier and Type	Method and Description
protected abstract String	counterGroup()
protected Map<String,String>	getDsTypeMap(org.apache.hadoop.mapreduce.Reducer.Context context, Set<String> dsTypeWhitelist)
protected com.google.common.base.Function<ImmutableBytesWritable,eu.dnetlib.data.proto.OafProtos.Oaf>	oafDeserialiser()
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context)
protected float	similarity(eu.dnetlib.data.proto.OafProtos.Oaf oa, eu.dnetlib.data.proto.OafProtos.Oaf ob)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

cleanup, reduce, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dedupConf

protected eu.dnetlib.pace.config.DedupConfig dedupConf

LIMIT

protected static final int LIMIT

See Also:

Constant Field Values

SCORE_DECIMALS

protected static final int SCORE_DECIMALS

See Also:

Constant Field Values

dsTypeMap

protected Map<String,String> dsTypeMap

dsWhitelist

protected Set<String> dsWhitelist

dsBlacklist

protected Set<String> dsBlacklist

untrustedOaDsList

protected Set<String> untrustedOaDsList

dsTypeWhitelist

protected Set<String> dsTypeWhitelist

scaleLB

protected double scaleLB

lower bound for trust scaling

Constructor Detail

AbstractEnrichmentReducer

public AbstractEnrichmentReducer()

Method Detail

counterGroup

protected abstract String counterGroup()

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<ImmutableBytesWritable,ImmutableBytesWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

getDsTypeMap

protected Map<String,String> getDsTypeMap(org.apache.hadoop.mapreduce.Reducer.Context context,
                                          Set<String> dsTypeWhitelist)
                                   throws IOException

Throws:

IOException

similarity

protected float similarity(eu.dnetlib.data.proto.OafProtos.Oaf oa,
                           eu.dnetlib.data.proto.OafProtos.Oaf ob)

oafDeserialiser

protected com.google.common.base.Function<ImmutableBytesWritable,eu.dnetlib.data.proto.OafProtos.Oaf> oafDeserialiser()