Scheduler¶

The `manila.scheduler.manager` Module¶

Scheduler Service

class SchedulerManager(scheduler_driver=None, service_name=None, *args, **kwargs)

Bases: manila.manager.Manager

Chooses a host to create shares.

RPC_API_VERSION = '1.11'

create_share_group(context, share_group_id, request_spec=None, filter_properties=None)

create_share_instance(context, request_spec=None, filter_properties=None)

create_share_replica(context, request_spec=None, filter_properties=None)

extend_share(context, share_id, new_size, reservations, request_spec=None, filter_properties=None)

get_host_list(context): Get a list of hosts from the HostManager.

get_pools(context, filters=None, cached=False): Get active pools from the scheduler’s cache.

get_service_capabilities(context): Get the normalized set of capabilities for this zone.

init_host_with_rpc(service_id=None)

A hook for service to do jobs after RPC is ready.

Like init_host(), this method is a hook where services get a chance to execute tasks that need RPC. Child classes should override this method.

Parameters: service_id – ID of the service where the manager is running.

manage_share(context, share_id, driver_options, request_spec, filter_properties=None): Ensure that the host exists and can accept the share.

migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec, filter_properties=None): Ensure that the host exists and can accept the share.

request_service_capabilities(context)

update_service_capabilities(context, service_name=None, host=None, capabilities=None, timestamp=None, **kwargs): Process a capability update from a service node.

The `manila.scheduler.base_handler` Module¶

A common base for handling extension classes.

Used by BaseFilterHandler and BaseWeightHandler

class BaseHandler(modifier_class_type, modifier_namespace)

Bases: object

Base class to handle loading filter and weight classes.

get_all_classes()

The `manila.scheduler.host_manager` Module¶

Manage hosts in the current zone.

class HostManager

Bases: object

Base HostManager class.

get_all_host_states_share(context, consider_disabled=False)

Returns a dict of all the hosts the HostManager knows about.

Each of the consumable resources in HostState are populated with capabilities scheduler received from RPC.

For example:: {‘192.168.1.100’: HostState(), …}

get_filtered_hosts(hosts, filter_properties, filter_class_names=None): Filter hosts and return only ones passing all filters.

get_pools(context, filters=None, cached=False): Returns a dict of all pools on all hosts HostManager knows about.

get_weighed_hosts(hosts, weight_properties, weigher_class_names=None): Weigh the hosts.

host_state_cls: alias of manila.scheduler.host_manager.HostState

update_service_capabilities(service_name, host, capabilities, timestamp): Update the per-service capabilities based on this notification.

class HostState(host, capabilities=None, service=None)

Bases: object

Mutable and immutable information tracked for a host.

consume_from_share(share): Incrementally update host state from an share.

update_backend(capability)

update_capabilities(capabilities=None, service=None)

update_from_share_capability(capability, service=None, context=None)

Update information about a host from its share_node info.

‘capability’ is the status info reported by share backend, a typical capability looks like this:

capability = {
    'share_backend_name': 'Local NFS',    #                'vendor_name': 'OpenStack',           #  backend level
    'driver_version': '1.0',              #  mandatory/fixed
    'storage_protocol': 'NFS',            #/ stats&capabilities

    'active_shares': 10,                  #                'IOPS_provisioned': 30000,            #  optional custom
    'fancy_capability_1': 'eat',          #  stats & capabilities
    'fancy_capability_2': 'drink',        #/

    'pools':[
      {
         'pool_name': '1st pool',               #                     'total_capacity_gb': 500,              #  mandatory stats
         'free_capacity_gb': 230,               #   for pools
         'allocated_capacity_gb': 270,          # |
         'qos': 'False',                        # |
         'reserved_percentage': 0,              # |
         'reserved_snapshot_percentage': 0,     # |
         'reserved_share_extend_percentage': 0, #/

         'dying_disks': 100,              #                     'super_hero_1': 'spider-man',    #  optional custom
         'super_hero_2': 'flash',         #  stats &
         'super_hero_3': 'neoncat',       #  capabilities
         'super_hero_4': 'green lantern', #/
       },
      {
         'pool_name': '2nd pool',
         'total_capacity_gb': 1024,
         'free_capacity_gb': 1024,
         'allocated_capacity_gb': 0,
         'qos': 'False',
         'reserved_percentage': 0,
         'reserved_snapshot_percentage': 0,
         'reserved_share_extend_percentage': 0,

         'dying_disks': 200,
         'super_hero_1': 'superman',
         'super_hero_2': 'Hulk',
      }]
}

update_pools(capability, service, context=None): Update storage pools information from backend reported info.

class PoolState(host, capabilities, pool_name)

Bases: manila.scheduler.host_manager.HostState

update_from_share_capability(capability, service=None, context=None): Update information about a pool from its share_node info.

update_pools(capability): Update storage pools information from backend reported info.

class ReadOnlyDict(source=None)

Bases: collections.UserDict

A read-only dict.

clear() → None. Remove all items from D.

pop(k[, d]) → v, remove specified key and return the corresponding value.: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair: as a 2-tuple; but raise KeyError if D is empty.

update([E, ]**F) → None. Update D from mapping/iterable E and F.: If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

The `manila.scheduler.rpcapi` Module¶

Client side of the scheduler manager RPC API.

class SchedulerAPI

Bases: object

Client side of the scheduler rpc API.

API version history:

1.0 - Initial version. 1.1 - Add get_pools method 1.2 - Introduce Share Instances. Replace create_share() with create_share_instance() 1.3 - Add create_consistency_group method (renamed in 1.7) 1.4 - Add migrate_share_to_host method 1.5 - Add create_share_replica 1.6 - Add manage_share 1.7 - Updated migrate_share_to_host method with new parameters 1.8 - Rename create_consistency_group -> create_share_group method 1.9 - Add cached parameter to get_pools method 1.10 - Add timestamp to update_service_capabilities 1.11 - Add extend_share

RPC_API_VERSION = '1.11'

create_share_group(context, share_group_id, request_spec=None, filter_properties=None)

Casts an rpc to the scheduler to create a share group.

Example of ‘request_spec’ argument value:

{

    'share_group_type_id': 'fake_share_group_type_id',
    'share_group_id': 'some_fake_uuid',
    'availability_zone_id': 'some_fake_az_uuid',
    'share_types': [models.ShareType],
    'resource_type': models.ShareGroup,

}

create_share_instance(context, request_spec=None, filter_properties=None)

create_share_replica(context, request_spec=None, filter_properties=None)

extend_share(context, share_id, new_size, reservations, request_spec, filter_properties=None)

get_pools(context, filters=None, cached=False)

manage_share(context, share_id, driver_options, request_spec=None, filter_properties=None)

migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec=None, filter_properties=None)

update_service_capabilities(context, service_name, host, capabilities)

The `manila.scheduler.scheduler_options` Module¶

SchedulerOptions monitors a local .json file for changes and loads it if needed. This file is converted to a data structure and passed into the filtering and weighing functions which can use it for dynamic configuration.

class SchedulerOptions

Bases: object

Monitor and load local .json file for filtering and weighing.

get_configuration(filename=None): Check the json file for changes and load it if needed.

The `manila.scheduler.drivers.base` Module¶

Scheduler base class that all Schedulers should inherit from

class Scheduler

Bases: object

The base class that all Scheduler classes should inherit from.

get_host_list(): Get a list of hosts from the HostManager.

get_pools(context, filters): Must override schedule method for scheduler to work.

get_service_capabilities(): Get the normalized set of capabilities for the services.

host_passes_filters(context, host, request_spec, filter_properties): Must override schedule method for migration to work.

hosts_up(context, topic): Return the list of hosts that have a running service for topic.

schedule(context, topic, method, *_args, **_kwargs): Must override schedule method for scheduler to work.

schedule_create_replica(context, request_spec, filter_properties): Must override schedule method for create replica to work.

schedule_create_share(context, request_spec, filter_properties): Must override schedule method for scheduler to work.

schedule_create_share_group(context, share_group_id, request_spec, filter_properties): Must override schedule method for scheduler to work.

update_service_capabilities(service_name, host, capabilities, timestamp): Process a capability update from a service node.

share_group_update_db(context, share_group_id, host)

Set the host and set the updated_at field of a share group.

Returns: A share group with the updated fields set properly.

share_replica_update_db(context, share_replica_id, host)

Set the host and the scheduled_at field of a share replica.

Returns: A Share Replica with the updated fields set.

share_update_db(context, share_id, host)

Set the host and set the scheduled_at field of a share.

Returns: A Share with the updated fields set properly.

The `manila.scheduler.drivers.chance` Module¶

Chance (Random) Scheduler implementation

class ChanceScheduler

Bases: manila.scheduler.drivers.base.Scheduler

Implements Scheduler as a random node selector.

schedule_create_share(context, request_spec, filter_properties): Picks a host that is up at random.

The `manila.scheduler.drivers.simple` Module¶

Simple Scheduler

class SimpleScheduler

Bases: manila.scheduler.drivers.chance.ChanceScheduler

Implements Naive Scheduler that tries to find least loaded host.

schedule_create_share(context, request_spec, filter_properties): Picks a host that is up and has the fewest shares.

Scheduler Filters¶

The `manila.scheduler.filters.availability_zone` Filter¶

class AvailabilityZoneFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filters Hosts by availability zone.

host_passes(host_state, filter_properties)

Return True if the HostState passes the filter, otherwise False.

Override this in a subclass.

run_filter_once_per_request = True

The `manila.scheduler.filters.base` Filter¶

Filter support

class BaseFilter

Bases: object

Base class for all filter classes.

filter_all(filter_obj_list, filter_properties)

Yield objects that pass the filter.

Can be overridden in a subclass, if you need to base filtering decisions on all objects. Otherwise, one can just override _filter_one() to filter a single object.

run_filter_for_index(index)

Check if filter needs to be run for the “index-th” instance.

Return True if the filter needs to be run for the “index-th” instance in a request. Only need to override this if a filter needs anything other than “first only” or “all” behaviour.

run_filter_once_per_request = False

class BaseFilterHandler(modifier_class_type, modifier_namespace)

Bases: manila.scheduler.base_handler.BaseHandler

Base class to handle loading filter classes.

This class should be subclassed where one needs to use filters.

get_filtered_objects(filter_classes, objs, filter_properties, index=0)

Get objects after filter

Parameters

filter_classes – filters that will be used to filter the objects
objs – objects that will be filtered
filter_properties – client filter properties
index – This value needs to be increased in the caller function of get_filtered_objects when handling each resource.

The `manila.scheduler.filters.base_host` Filter¶

Scheduler host filters

class BaseHostFilter

Bases: manila.scheduler.filters.base.BaseFilter

Base class for host filters.

host_passes(host_state, filter_properties)

Return True if the HostState passes the filter, otherwise False.

Override this in a subclass.

class HostFilterHandler(namespace): Bases: manila.scheduler.filters.base.BaseFilterHandler

The `manila.scheduler.filters.capabilities` Filter¶

class CapabilitiesFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

HostFilter to work with resource (instance & volume) type records.

host_passes(host_state, filter_properties): Return a list of hosts that can create resource_type.

The `manila.scheduler.filters.capacity` Filter¶

class CapacityFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

CapacityFilter filters based on share host’s capacity utilization.

host_passes(host_state, filter_properties): Return True if host has sufficient capacity.

The `manila.scheduler.filters.extra_specs_ops` Filter¶

match(value, req)

The `manila.scheduler.filters.ignore_attempted_hosts` Filter¶

class IgnoreAttemptedHostsFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filter out previously attempted hosts

A host passes this filter if it has not already been attempted for scheduling. The scheduler needs to add previously attempted hosts to the ‘retry’ key of filter_properties in order for this to work correctly. For example:

{
 'retry': {
           'hosts': ['host1', 'host2'],
           'num_attempts': 3,
          }
}

host_passes(host_state, filter_properties): Skip nodes that have already been attempted.

The `manila.scheduler.filters.json` Filter¶

class JsonFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Host Filter to allow simple JSON-based grammar for selecting hosts.

commands = {'<': <function JsonFilter._less_than>, '<=': <function JsonFilter._less_than_equal>, '=': <function JsonFilter._equals>, '>': <function JsonFilter._greater_than>, '>=': <function JsonFilter._greater_than_equal>, 'and': <function JsonFilter._and>, 'in': <function JsonFilter._in>, 'not': <function JsonFilter._not>, 'or': <function JsonFilter._or>}

host_passes(host_state, filter_properties)

Filters hosts.

Return a list of hosts that can fulfill the requirements specified in the query.

The `manila.scheduler.filters.retry` Filter¶

class RetryFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filter out already tried nodes for scheduling purposes.

host_passes(host_state, filter_properties): Skip nodes that have already been attempted.

Scheduler Weighers¶

The `manila.scheduler.weighers.base` Weigher¶

Pluggable Weighing support

class BaseWeigher

Bases: object

Base class for pluggable weighers.

The attributes maxval and minval can be specified to set up the maximum and minimum values for the weighed objects. These values will then be taken into account in the normalization step, instead of taking the values from the calculated weighers.

maxval = None

minval = None

weigh_objects(weighed_obj_list, weight_properties)

Weigh multiple objects.

Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.

weight_multiplier()

How weighted this weigher should be.

Override this method in a subclass, so that the returned value is read from a configuration option to permit operators specify a multiplier for the weigher.

class BaseWeightHandler(modifier_class_type, modifier_namespace)

Bases: manila.scheduler.base_handler.BaseHandler

get_weighed_objects(weigher_classes, obj_list, weighing_properties): Return a sorted (descending), normalized list of WeighedObjects.

object_class: alias of manila.scheduler.weighers.base.WeighedObject

class WeighedObject(obj, weight)

Bases: object

Object with weight information.

normalize(weight_list, minval=None, maxval=None)

Normalize the values in a list between 0 and 1.0.

The normalization is made regarding the lower and upper values present in weight_list. If the minval and/or maxval parameters are set, these values will be used instead of the minimum and maximum from the list.

If all the values are equal, they are normalized to 0.

The `manila.scheduler.weighers.base_host` Weigher¶

Scheduler host weighers

class BaseHostWeigher

Bases: manila.scheduler.weighers.base.BaseWeigher

Base class for host weighers.

class HostWeightHandler(namespace)

Bases: manila.scheduler.weighers.base.BaseWeightHandler

object_class: alias of manila.scheduler.weighers.base_host.WeighedHost

class WeighedHost(obj, weight)

Bases: manila.scheduler.weighers.base.WeighedObject

to_dict()

The `manila.scheduler.weighers.capacity` Weigher¶

Capacity Weigher. Weigh hosts by their virtual or actual free capacity.

For thin provisioning, weigh hosts by their virtual free capacity calculated by the total capacity multiplied by the max over subscription ratio and subtracting the provisioned capacity; Otherwise, weigh hosts by their actual free capacity, taking into account the reserved space.

The default is to spread shares across all hosts evenly. If you prefer stacking, you can set the ‘capacity_weight_multiplier’ option to a negative number and the weighing has the opposite effect of the default.

class CapacityWeigher

Bases: manila.scheduler.weighers.base_host.BaseHostWeigher

weigh_objects(weighed_obj_list, weight_properties)

Weigh multiple objects.

Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.

weight_multiplier(): Override the weight multiplier.

The `manila.scheduler.weighers.pool` Weigher¶

class PoolWeigher

Bases: manila.scheduler.weighers.base_host.BaseHostWeigher

weight_multiplier(): Override the weight multiplier.

Scheduler

Scheduler¶

The `manila.scheduler.manager` Module¶

The `manila.scheduler.base_handler` Module¶

The `manila.scheduler.host_manager` Module¶

The `manila.scheduler.rpcapi` Module¶

The `manila.scheduler.scheduler_options` Module¶

The `manila.scheduler.drivers.filter` Module¶

The `manila.scheduler.drivers.base` Module¶

The `manila.scheduler.drivers.chance` Module¶

The `manila.scheduler.drivers.simple` Module¶

Scheduler Filters¶

The `manila.scheduler.filters.availability_zone` Filter¶

The `manila.scheduler.filters.base` Filter¶

The `manila.scheduler.filters.base_host` Filter¶

The `manila.scheduler.filters.capabilities` Filter¶

The `manila.scheduler.filters.capacity` Filter¶

The `manila.scheduler.filters.extra_specs_ops` Filter¶

The `manila.scheduler.filters.ignore_attempted_hosts` Filter¶

The `manila.scheduler.filters.json` Filter¶

The `manila.scheduler.filters.retry` Filter¶

Scheduler Weighers¶

The `manila.scheduler.weighers.base` Weigher¶

The `manila.scheduler.weighers.base_host` Weigher¶

The `manila.scheduler.weighers.capacity` Weigher¶

The `manila.scheduler.weighers.pool` Weigher¶

manila 19.0.1

Page Contents

Scheduler

Scheduler¶

The manila.scheduler.manager Module¶

The manila.scheduler.base_handler Module¶

The manila.scheduler.host_manager Module¶

The manila.scheduler.rpcapi Module¶

The manila.scheduler.scheduler_options Module¶

The manila.scheduler.drivers.filter Module¶

The manila.scheduler.drivers.base Module¶

The manila.scheduler.drivers.chance Module¶

The manila.scheduler.drivers.simple Module¶

Scheduler Filters¶

The manila.scheduler.filters.availability_zone Filter¶

The manila.scheduler.filters.base Filter¶

The manila.scheduler.filters.base_host Filter¶

The manila.scheduler.filters.capabilities Filter¶

The manila.scheduler.filters.capacity Filter¶

The manila.scheduler.filters.extra_specs_ops Filter¶

The manila.scheduler.filters.ignore_attempted_hosts Filter¶

The manila.scheduler.filters.json Filter¶

The manila.scheduler.filters.retry Filter¶

The manila.scheduler.filters.share_replication Filter¶

Scheduler Weighers¶

The manila.scheduler.weighers.base Weigher¶

The manila.scheduler.weighers.base_host Weigher¶

The manila.scheduler.weighers.capacity Weigher¶

The manila.scheduler.weighers.pool Weigher¶

manila 19.0.1

Page Contents

The `manila.scheduler.manager` Module¶

The `manila.scheduler.base_handler` Module¶

The `manila.scheduler.host_manager` Module¶

The `manila.scheduler.rpcapi` Module¶

The `manila.scheduler.scheduler_options` Module¶

The `manila.scheduler.drivers.filter` Module¶

The `manila.scheduler.drivers.base` Module¶

The `manila.scheduler.drivers.chance` Module¶

The `manila.scheduler.drivers.simple` Module¶

The `manila.scheduler.filters.availability_zone` Filter¶

The `manila.scheduler.filters.base` Filter¶

The `manila.scheduler.filters.base_host` Filter¶

The `manila.scheduler.filters.capabilities` Filter¶

The `manila.scheduler.filters.capacity` Filter¶

The `manila.scheduler.filters.extra_specs_ops` Filter¶

The `manila.scheduler.filters.ignore_attempted_hosts` Filter¶

The `manila.scheduler.filters.json` Filter¶

The `manila.scheduler.filters.retry` Filter¶

The `manila.scheduler.filters.share_replication` Filter¶

The `manila.scheduler.weighers.base` Weigher¶

The `manila.scheduler.weighers.base_host` Weigher¶

The `manila.scheduler.weighers.capacity` Weigher¶

The `manila.scheduler.weighers.pool` Weigher¶