Unfortunately there is no setting for this. We designed the product pricing system for the "lowest common denominator" of always recalculating on load, but in 2.x we'll make sure it's more pluggable to tweak for performance. You could use conditions in your pricing rules themselves to check the page the user is currently viewing before executing a pricing action, which should prevent any unnecessary recalculation triggered saves earlier on. But that still won't prevent loads / checking.

The only alternative is to write a module that basically overrides the default hook_commerce_order_load() the Cart module uses to refresh shopping cart orders. Using Drupal's hook_module_implements_alter() you should be able to unset the implementation registered for the Cart module. Then your custom module can implement the same order load hook and only call commerce_cart_commerce_order_load() when you want it to.

Again, less than ideal, but a high priority feature to address in Commerce 2.x.