thredds.inventory

Class MFileCollectionManager

java.lang.Object

thredds.inventory.CollectionAbstract

thredds.inventory.CollectionManagerAbstract

thredds.inventory.MFileCollectionManager

All Implemented Interfaces:

AutoCloseable, CollectionManager, CollectionUpdateListener, MCollection

@ThreadSafe
public class MFileCollectionManager
extends CollectionManagerAbstract

Manage Collections of MFiles. Used in:

• replaces older version in ncml.Aggregation
• ucar.nc2.ft.point.collection.TimedCollectionImpl (ucar.nc2.ft.point.collection.CompositeDatasetFactory)
• ucar.nc2.ft.fmrc.Fmrc
• ucar.nc2.grib.GribCollection

we need to be thread safe, for InvDatasetFeatureCollection

Since:

Jul 8, 2009

Author:

caron

Nested Class Summary

Nested classes/interfaces inherited from class thredds.inventory.CollectionAbstract

CollectionAbstract.DateSorter, CollectionAbstract.MyGribFilter

Nested classes/interfaces inherited from interface thredds.inventory.CollectionManager

CollectionManager.ChangeChecker, CollectionManager.TriggerEvent, CollectionManager.TriggerListener

Field Summary

Fields

Modifier and Type	Field and Description
protected FeatureCollectionConfig	config

Fields inherited from class thredds.inventory.CollectionManagerAbstract

changeChecker, recheck

Fields inherited from class thredds.inventory.CollectionAbstract

auxInfo, CATALOG, collectionName, dateExtractor, DIR, FILE, filter, GLOB, lastModified, LIST, logger, NCX_SUFFIX, protoChoice, root, sfilter, startCollection

Constructor Summary

Constructors

Modifier	Constructor and Description
	MFileCollectionManager(FeatureCollectionConfig config, Formatter errlog, org.slf4j.Logger logger)
	MFileCollectionManager(String name, CollectionConfig mc, CalendarDate startPartition, org.slf4j.Logger logger)
protected	MFileCollectionManager(String name, org.slf4j.Logger logger)
	MFileCollectionManager(String name, String spec, Formatter errlog, org.slf4j.Logger logger)

Method Summary

Methods

Modifier and Type	Method and Description
void	addDirectoryScan(String dirName, String suffix, String regexpPatternString, String subdirsS, String olderS, Object auxInfo) Add a directory scan to the collection
static MController	getController()
Iterable<MFile>	getFilesSorted() Get the current collection of MFile.
long	getLastChanged() Get the last time the collection changed
long	getLastScanned() Get the last time scanned
long	getOlderThanFilterInMSecs()
boolean	hasDateExtractor() Does this CollectionManager have the ability to extract a date from the MFile ?
protected boolean	hasScans()
boolean	isScanNeeded() Compute if synchronous scan is needed.
static MFileCollectionManager	open(String collectionName, String collectionSpec, String olderThan, Formatter errlog)
static MFileCollectionManager	openWithRecheck(String collectionName, String recheckS)
protected void	reallyScan(Map<String,MFile> map)
boolean	scan(boolean sendEvent) Do not use
static void	setController(MController _controller) Set the MController used by scan.
void	setFiles(Iterable<MFile> files)
String	toString()

Methods inherited from class thredds.inventory.CollectionManagerAbstract

addEventListener, close, createListenerManager, getFileIterator, getMetadata, getRecheck, isStatic, putMetadata, removeEventListener, scanIfNeeded, sendEvent, setChangeChecker, setMetadataStore, setStatic

Methods inherited from class thredds.inventory.CollectionAbstract

cleanName, extractDate, getAuxInfo, getCollectionName, getFilenames, getIndexFilename, getLastModified, getLatestFile, getProtoIndex, getRoot, getStartCollection, isLeaf, makeFileListSorted, parseOlderThanString, putAuxInfo, setDateExtractor, setLeaf, setMFileFilter, setRoot, setStreamFilter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface thredds.inventory.MCollection

extractDate, getAuxInfo, getCollectionName, getFilenames, getIndexFilename, getLastModified, getLatestFile, getProtoIndex, getRoot, getStartCollection, isLeaf, putAuxInfo

Methods inherited from interface thredds.inventory.CollectionUpdateListener

getCollectionName

Field Detail

config

protected FeatureCollectionConfig config

Constructor Detail

MFileCollectionManager

public MFileCollectionManager(FeatureCollectionConfig config,
                      Formatter errlog,
                      org.slf4j.Logger logger)

MFileCollectionManager

protected MFileCollectionManager(String name,
                      org.slf4j.Logger logger)

MFileCollectionManager

public MFileCollectionManager(String name,
                      String spec,
                      Formatter errlog,
                      org.slf4j.Logger logger)

MFileCollectionManager

public MFileCollectionManager(String name,
                      CollectionConfig mc,
                      CalendarDate startPartition,
                      org.slf4j.Logger logger)

Method Detail

setController

public static void setController(MController _controller)

Set the MController used by scan. Defaults to thredds.filesystem.ControllerOS() if not set.

Parameters:

_controller - use this MController

getController

public static MController getController()

open

public static MFileCollectionManager open(String collectionName,
                          String collectionSpec,
                          String olderThan,
                          Formatter errlog)
                                   throws IOException

Throws:

IOException

openWithRecheck

public static MFileCollectionManager openWithRecheck(String collectionName,
                                     String recheckS)

addDirectoryScan

public void addDirectoryScan(String dirName,
                    String suffix,
                    String regexpPatternString,
                    String subdirsS,
                    String olderS,
                    Object auxInfo)

Add a directory scan to the collection

Parameters:

dirName - scan this directory

suffix - require this suffix (overriddden by regexp), may be null

regexpPatternString - if present, use this reqular expression to filter files , may be null

subdirsS - if "true", descend into subdirectories, may be null

olderS - udunit time unit - files must be older than this amount of time (now - lastModified > olderTime), may be null // * @param dateFormatString dateFormatMark string, may be null

auxInfo - attach this object to any MFile found by this scan

getOlderThanFilterInMSecs

public long getOlderThanFilterInMSecs()

getLastScanned

public long getLastScanned()

Description copied from interface: CollectionManager

Get the last time scanned

Returns:

msecs since 1970

getLastChanged

public long getLastChanged()

Description copied from interface: CollectionManager

Get the last time the collection changed

Returns:

msecs since 1970

hasScans

protected boolean hasScans()

isScanNeeded

public boolean isScanNeeded()

Compute if synchronous scan is needed. True if recheck is true and enough time has elapsed.

Returns:

true if rescan is needed

scan

public boolean scan(boolean sendEvent)
             throws IOException

Do not use

Returns:

true if anything actually changed.

Throws:

IOException - public void scanDebug(Formatter f) throws IOException { getController(); // make sure a controller is instantiated // run through all scanners and collect MFile instances into the Map for (CollectionConfig mc : scanList) { // lOOK: are there any circumstances where we dont need to recheck against OS, ie always use cached values? Iterator iter = (mc.wantSubdirs()) ? controller.getInventoryAll(mc, true) : controller.getInventoryTop(mc, true); /// NCDC wants subdir /global/nomads/nexus/gfsanl..gfsanl_3_.*\.grb$ if (iter == null) { logger.error(collectionName + ": Invalid collection= " + mc); continue; } int count = 0; while (iter.hasNext()) { MFile mfile = iter.next(); mfile.setAuxInfo(mc.getAuxInfo()); map.put(mfile.getPath(), mfile); count++; } logger.debug("{} : was scanned nfiles= {} ", collectionName, count); } }

setFiles

public void setFiles(Iterable<MFile> files)

getFilesSorted

public Iterable<MFile> getFilesSorted()

Description copied from interface: MCollection

Get the current collection of MFile. if hasDateExtractor() == true, these will be sorted by Date, otherwise by path.

Returns:

current collection of MFile as an Iterable.

hasDateExtractor

public boolean hasDateExtractor()

Description copied from interface: MCollection

Does this CollectionManager have the ability to extract a date from the MFile ?

Specified by:

hasDateExtractor in interface MCollection

Overrides:

hasDateExtractor in class CollectionAbstract

Returns:

true if CollectionManager has a DateExtractor

reallyScan

protected void reallyScan(Map<String,MFile> map)
                   throws IOException

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object