No, adding Dispatch From and Dispatch To addresses is not mandatory while creating an E-invoice in most cases. However, these details may be required in certain scenarios based on the nature of the transaction.
If you want to create E-way bill for your E-invoice then those details are mandatory.