Harper's minority government could topple if the Liberal Opposition voted with the NDP and Bloc Quebecois against a confidence motion.
But Harper told Conservative members at a Stampede barbecue that he won't be held hostage and will continue to push legislation that lowers taxes and gets tough on crime.
Harper also touted his government's success in lowering taxes even through a global recession and warned that with any of the other political parties in power the trend would not continue.
Liberal Leader Michael Ignatieff also spoke in Calgary earlier in the day as part of a Stampede breakfast.
He blasted Harper for negative attack ads and said the Liberals are presenting a solid alternative for people fed up with the way the Tories are running the country.