When defining affect, many will go back to Spinoza, the capacity for being affected. More useful perhaps is this from Aristotle which I think should be read in tandem with this bit on potentiality. The first puts the emphasis on alteration, the second on change and movement.
“Affection means  a quality in respect of which a thing can be altered, e.g. white and black, sweet and bitter, heaviness and lightness, and others of this kind —(2), The actualization of these –the already accomplished alterations.—(3) Especially injurious alterations and movements, and, above all, painful injuries. –-(4) Misfortunes and painful experiences when on a large scale ae are called affections.” (Metaphysics 1022b)
“I mean by potency not only that definite kind which is said to be a principle of change in another thing or in the thing itself regarded as other, but in general every principle of movement or of rest. For nature is in the same genus as potency; for it is a principle of movement –not, however, in something else but in the thing itself qua itself. ” (Metaphysics 1049b)
(translated by W.D. Ross)