Flexible Staffing for Call Centers with Non-stationary Arrival Rates

Abstract

We consider a multi-period staffing problem of a single-skill call center. The call center is modeled as a multi-server queue in which the staffing levels can be changed only at specific moments in time. The objective is to set the staffing levels such that a service level constraint is met in the presence of time-varying arrival rates. We develop a Markov decision model to obtain time-dependent staffing levels for both the case where the arrival rate function is known as well as unknown. The characteristics of the optimal policies associated to the two cases are illustrated through a numerical study based on real-life data. We show that the optimal policies provide a good balance between staffing costs and the penalty probability for not meeting the service level.

Appendix: Exact Solution

In this section, we formulate a discrete-time Markov decision problem for our original continuous-time problem. We only discretize time into small intervals, but make no other approximation. Hence, the formulation is nearly exact for small time intervals, and thus computes nearly optimal policies for the original problem. We denote the length of a time interval by 1∕η, thus every 1∕η time units the system is observed.

In order to model the transitions of the system after each observation, we need a large state space that contains all information to calculate the next state. Hence, define the state space $\mathcal{X}$ to consist of tuples (n, s _c , s _d , m, z, w ₁, …, w _n ). In this tuple, $n \in \mathbb{N}_{0}$ denotes the number of customers in the system at the time of an observation. The realized waiting times of each of the n customers at the moment of observation is given by $w_{1},\mathop{\ldots },w_{n} \in \mathbb{R}_{0}^{+}$. We will adopt the convention that customers in service have a waiting time of 0. Further, let $s_{c} \in \mathbb{N}_{0}$ denote the number of servers currently in use, and $s_{d} \in \mathbb{N}_{0}$ the number of servers that is desired to have. The service level can be computed by the ratio of $z \in \mathbb{N}_{0}$, the number of customers served within τ time units, and $m \in \mathbb{N}_{0}$, the number of customers served. These variables are sufficient to model the state transitions in a Markovian way. Hence, the dynamic programming backward recursion formula becomes

$$\displaystyle\begin{array}{rcl} & & \eta V _{k+1}(n,s_{c},s_{d},m,z,w_{1},\ldots,w_{n}) = c_{1}s_{k} + c_{2}(s_{c} - s_{k}) {}\\ & & \quad +\lambda \mathbb{1} _{\{s_{c}<n\}}H_{k}(n + 1,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{s_{c}},w_{s_{c}+1} + 1/\eta,\mathop{\ldots },w_{n} + 1/\eta,0) {}\\ & & \quad +\lambda \mathbb{1} _{\{s_{c}=n\}}H_{k}(n + 1,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{n},0) {}\\ & & \quad +\lambda \mathbb{1} _{\{s_{c}>n\}}H_{k}(n + 1,s_{c},s_{d},m + 1,z + 1,w_{1},\mathop{\ldots },w_{n},0) {}\\ & & \quad +\mu s_{c} \mathbb{1} _{\{s_{c}=s_{d}\}} \mathbb{1} _{\{s_{c}<n\}}\big[H_{k}(n - 1,s_{c},s_{d},m + 1,z+ \mathbb{1} _{\{w_{s_{ c}+1}+1/\eta <\tau \}}, {}\\ & & \qquad w_{2},\mathop{\ldots },w_{s_{c}},0,w_{s_{c}+2} + 1/\eta,\mathop{\ldots },w_{n} + 1/\eta )\big] {}\\ & & \quad +\mu s_{c} \mathbb{1} _{\{s_{c}>s_{d}\}} \mathbb{1} _{\{s_{c}<n\}}\big[H_{k}(n - 1,s_{c} - 1,s_{d},m,z, {}\\ & & \qquad w_{2},\mathop{\ldots },w_{s_{c}},w_{s_{c}+1} + 1/\eta,\mathop{\ldots },w_{n} + 1/\eta )\big] {}\\ & & \quad +\mu n \mathbb{1} _{\{s_{c}=s_{d}\}} \mathbb{1} _{\{s_{c}\geq n\}}\big[H_{k}(n - 1,s_{c},s_{d},m,z,w_{2},\mathop{\ldots },w_{n})\big] {}\\ & & \quad +\mu n \mathbb{1} _{\{s_{c}>s_{d}\}} \mathbb{1} _{\{s_{c}\geq n\}}\big[H_{k}(n - 1,s_{c} - 1,s_{d},m,z,w_{2},\mathop{\ldots },w_{n})\big] {}\\ & & \quad +\big (\eta -\lambda -\text{min}\{n,s_{c}\}\mu \big)\big[ \mathbb{1} _{\{s_{c}\geq n\}}H_{k}(n,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{n}) {}\\ & & \quad + \mathbb{1} _{\{s_{c}<n\}}H_{k}(n,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{s_{c}},w_{s_{c}+1} + 1/\eta,\mathop{\ldots },w_{n} + 1/\eta )\big]. {}\\ \end{array}$$

The index k counts the number of intervals to go until the end of the complete period, the last interval. The first two terms describe the cost of using s _k permanent and s _c − s _k flexible agents. If upon arrival, the number of servers currently in use is less than n, then there are n − s _c customers in the queue. Hence, these customers add 1∕η time units to their waiting time (term 3). If s _c  = n, then everyone is in service, and the arriving customer has to wait (term 4). If s _c  > n, then there are idle servers. Hence, an arriving customer is served immediately and satisfies the service level directly as well (term 5). The next two terms, terms 6 and 7, model the case where a customer leaves the system when there are customers waiting in the queue. The first case is where the number of servers currently in use is equal to the desired number. Hence, 1∕η is added to the waiting times and s _c remains unchanged. When the customer is taken into service, then the service level is also adjusted. The second case is when s _c is higher than s _d , then additionally s _c is decreased by one and no customer is taken into service (thus, the service level is not updated either). Terms 8 and 9 model a similar situation, however, in this case there are a sufficient number of servers available so that no customer is waiting. Hence, the waiting times are not adjusted and neither is the service level. The final terms deal with the similar cases in which no event occurs within the interval. Hence, only the waiting times are updated when s _c  < n.

In the dynamic programming backward recursion formula, we have adopted the notation H for the action operator. This action operator is equal to V for all intervals k that are not a decision epoch, i.e., $k\notin \mathcal{T}$. However, for all $k \in \mathcal{T}$, we have that

$$\displaystyle\begin{array}{rcl} H_{k}(n,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{n})& =& \text{min}_{l\in \mathbb{N}_{0}}\big\{\{V _{k}(n,s_{c},l,m,z,w_{1},\mathop{\ldots },w_{n})\,\vert \,l <s_{c}\} {}\\ & & \cup \{V _{k}(n,l,l,m,z,w_{1},\mathop{\ldots },w_{n})\,\vert \,l \geq s_{c}\}\big\}. {}\\ \end{array}$$

The first set in the minimization models the case in which the number of servers is decreased, hence s _d is adjusted. The second set models the case in which the number of servers is increased. Since this happens immediately, both s _c and s _d are set to the desired level.

Finally, we finish the model by describing what happens at the last interval. In this case, we can evaluate the realized service level and compare it to α. If the service level is not met, then a penalty of P is incurred, and otherwise no additional cost is incurred. This is given by the following equation.

$$\displaystyle{ \eta V _{0}(n,s_{c},s_{d},m,z,w_{1},\mathop{\ldots },w_{n}) = c_{1}s_{k} + c_{2}(s_{c} - s_{k}) + P \mathbb{1} _{\{z/m<\alpha \}}. }$$