us.codecraft.webmagic.scheduler.component

Class HashSetDuplicateRemover

java.lang.Object

us.codecraft.webmagic.scheduler.component.HashSetDuplicateRemover

All Implemented Interfaces:

DuplicateRemover

public class HashSetDuplicateRemover
extends java.lang.Object
implements DuplicateRemover

Author:

code4crafer@gmail.com

Constructor Summary

Constructors

Constructor and Description
HashSetDuplicateRemover()

Method Summary

Modifier and Type	Method and Description
int	getTotalRequestsCount(Task task) Get TotalRequestsCount for monitor.
protected java.lang.String	getUrl(Request request)
boolean	isDuplicate(Request request, Task task) Check whether the request is duplicate.
void	resetDuplicateCheck(Task task) Reset duplicate check.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HashSetDuplicateRemover

public HashSetDuplicateRemover()

Method Detail

isDuplicate

public boolean isDuplicate(Request request,
                           Task task)

Description copied from interface: DuplicateRemover

Check whether the request is duplicate.

Specified by:

isDuplicate in interface DuplicateRemover

Parameters:

request - request

task - task

Returns:

true if is duplicate

getUrl

protected java.lang.String getUrl(Request request)

resetDuplicateCheck

public void resetDuplicateCheck(Task task)

Description copied from interface: DuplicateRemover

Reset duplicate check.

Specified by:

resetDuplicateCheck in interface DuplicateRemover

Parameters:

task - task

getTotalRequestsCount

public int getTotalRequestsCount(Task task)

Description copied from interface: DuplicateRemover

Get TotalRequestsCount for monitor.

Specified by:

getTotalRequestsCount in interface DuplicateRemover

Parameters:

task - task

Returns:

number of total request