train.ConcatExpertChunkedSampler Class Reference

Inheritance diagram for train.ConcatExpertChunkedSampler:

Collaboration diagram for train.ConcatExpertChunkedSampler:

Public Member Functions
	__init__ (self, expert_datasets, chunk_size=1_000_000)
	__iter__ (self)
	__len__ (self)

Public Attributes
	chunk_size = chunk_size
list	shard_offsets = []
list	spatial_chunks = []
	num_samples = total

Detailed Description

Expert sampler for ConcatDataset preserving per-shard spatial locality.

Constructor & Destructor Documentation

__init__()

train.ConcatExpertChunkedSampler.__init__	(		self,
			expert_datasets,
			chunk_size = 1_000_000 )

Member Function Documentation

__iter__()

train.ConcatExpertChunkedSampler.__iter__

(

self

)

__len__()

train.ConcatExpertChunkedSampler.__len__

(

self

)

Member Data Documentation

chunk_size

train.ConcatExpertChunkedSampler.chunk_size = chunk_size

num_samples

train.ConcatExpertChunkedSampler.num_samples = total

shard_offsets

list train.ConcatExpertChunkedSampler.shard_offsets = []

spatial_chunks

list train.ConcatExpertChunkedSampler.spatial_chunks = []

---

The documentation for this class was generated from the following file:

• training/train.py