Unlike the real part of the generalized weak value of an observable, which can in a restricted sense be operationally interpreted as an idealized conditioned average of that observable in the limit of zero measurement disturbance, the imaginary part of the generalized weak value does not provide information pertaining to the observable being measured. What it does provide is direct information about how the initial state would be unitarily disturbed by the observable operator. Specifically, we provide an operational interpretation for the imaginary part of the generalized weak value as the logarithmic directional derivative of the post-selection probability along the unitary flow generated by the action of the observable operator. To obtain this interpretation, we revisit the standard von Neumann measurement protocol for obtaining the real and imaginary parts of the weak value and solve it exactly for arbitrary initial states and post-selections using the quantum operations formalism, which allows us to understand in detail how each part of the generalized weak value arises in the linear response regime. We also provide exact treatments of qubit measurements and Gaussian detectors as illustrative special cases, and show that the measurement disturbance from a Gaussian detector is purely decohering in the Lindblad sense, which allows the shifts for a Gaussian detector to be completely understood for any coupling strength in terms of a single complex weak value that involves the decohered initial state.