There is a recent trend among agencies to shift the legality requirement to the copyright holder who is sending the work to the agent, aka the photographer.
A lot of this is practical. The agent cannot certify that a model/property release is legal since the agent was not a witness to the proceedings of how it was obtained (proper signature, compensation, etc). Then there are trademark issues (as products get super savvy to commercial licensing globally) for the items contained in the images. For example photographers licensing stock pictures, or any commercial photo, cannot use identifiable Le Corbusier designed furniture (such as knock-off Barcelona chairs) nor those funny little air freshener pine trees in rear view mirrors (the kind that read "road trip" Americana). That's just the start. This does not affect only the stock photo. If you're shooting a Coca-Cola ad, you wouldn't want a 2 liter bottle sitting atop a Le Corbusier designed table without a proper rights clearance and that is difficult to obtain.
So the conclusion is that is it the copyright holder's responsibility to take control for all of these legalities and indemnify the agent that is charged with selling the work. If you sell the work to the agent and transfer the copyright in the process, then the photographer is no longer responsible. This is only in theory because this is not really an option anymore. No stock photo agency that I know of is buying content.
It all sounds a bit open-ended legally, so let's break it down. You can assist your legal protection with E&O (Errors and Omissions) Insurance. It's not iron-clad, but it's there to help as a tack-on to an already existing business policy for photography. This does make it a bit difficult, because if you don't have the business insurance policy, you cannot get the E&O. Figure $450 per year to get started for that as a package in entirety. E&O on its own is about $150 as an add-on.
As you can see, the investment starts to add up. Will you be licensing enough images to justify the $150 E&O? It's a question that many are asking, because agencies themselves do not offer this insurance to the photographers. It's answered perhaps by looking at another mega-trend: microstock. Are the 150,000 or so contributors that make up a microstock agency community all indemnifying themselves with E&O?...certainly not. They're winging it; and there's probably nothing wrong with that. It's a practical matter.
Remember, most agencies will not take images (even with a release attached) that they deem even slightly, remotely, or possibly construed as violations of the trademark of an object/item/product in the picture. This means that all barcodes, insignia, logos, shapes, etc of items must be altered. It takes great care and time in post-production retouching to handle this. There's almost an art to it. Expect to spend a great deal of time scouring images at 100% on screen to make sure that everything is removed before submission. If it is out of focus, but still identifiable, it will be rejected. All of this, protects you from the start.
The other pragmatic component to look at is this: what is in the image that one would come back legally to pursue you about? Models have to be released and agencies will not accept just any release. This release will protect you; and if a model does come back to complain, then the simplest recourse is to take down the photo from the agency selling it. Problem solved. Ditto for property issues. As for the defamatory use problem, well, that's always out of the photographer's control. I can understand your worries. Every agency has an End-User-License-Agreement (EULA) that the purchaser of the image must agree to, even if they don't uphold its terms (which in all practically are not checked, but understood to be followed). If the end user (client) of the image violates the terms of the use of the image, the photographer is not held liable. The photographer did his/her best practice to provide the image to the agency legally, and the agent sold it with best practice legally. Defamatory use would then be caught by this EULA agreement and it would be the responsibility of the client to make amends to the violation.
Please remember that I'm speaking in broad terms here from practical down-to-earth perspectives of how these things are handled. If you are really concerned, it's always best to consult legal advice in a case by case situation.