However, there seems to be a significant issue in Magento's Paging code when you attempt to use a collection as a source for deleting items. Let's look at a piece of example code:

$myCollection = Mage::getModel('test/model')->getCollection()->setPageSize(10);
for($page = 1; $page <= $myCollection->getLastPageNumber(); $page++)
{
    $myCollection->clear()->setCurPage($page)->load();
    //delete all items on this page
    foreach($myCollection->getItems() as $_item)
    {
        $_item->delete();
    }
}

What this code does is load up a collection, tell it that each page should only load 10 items at a time, and then attempt to loop through each page. Unfortunately, the collection object does not have a built-in Iterator for pages themselves, or for items not on the current page, which could alleviate the issue we're discussing.

The problem here is that collections in Magento are lazy-loaded, which is a good thing: it means that unnecessary data is left behind, and allows for your memory usage to remain low on high-count operations if you split the operations properly. Each time load() is called above, the following SQL is called:

SELECT * FROM test_model_item_table LIMIT 10 OFFSET (calculated_offset_start)

"calculated_offset_start" is a variable determined by Magento to be the start of the next page and sent into the SQL query. Those with a keen eye for bad SQL will probably see the issue immediately:

The Collection's current page offset calculator does not account for deleted data, because the collection itself doesn't manage the data. While this is technically not a failing of the collection object, it causes a massive gotcha when deletes are done during the iteration. This is enhanced by an actual bug in the collection, however, because it does not recalculate the value of getLastPageNumber(), so even if you use that as a guiding point, you will receive an incorrect number of pages.

The workaround to handle this is to always use Page 1 until you know that all operations have been performed, even on new entries to that page. The Collection's walk function only addresses items within the currently loaded page, so it is unfortunately not an option.

Hopefully at some point in the future, the Collection object will build in a Page iterator to better handle these issues. The ideal solution would be for aggregate functions to be available in the Collection itself, so it is not completely blind to the data inside. I hope this post helped any strange collection deletion bugs you were having!

SQL Error 150 - Can't create table ....

You might run into this error when Magento is trying to create tables with foreign key assignments to tables that do not yet exist. When this happens find the line in the setup script triggering that error and surround it with the following commands:

$installer->run("
 set foreign_key_checks = 0 ;"
);


.... other installer code ...


$installer->run("
 set foreign_key_checks = 1 ;"
);

Then, trigger the installation again and after successful execution remove the modifications.

Code Changes

• If you extend / overwrote Customer/controllers/AccountController.php, you must update any code involving the Forgot Password system. It has been changed considerably. Your code now must call a different function to send the email, and must include the code to generate an authorization token for reset.
• If your changes involve latching onto the Forgot Password events, be sure to update them accordingly to listen for the new Reset Password actions.

Configuration Changes

Forgot Password Email Template Assignment:

The template set for "Forgot Password" under System > Configuration > Customers:Customer Configuration must be changed to a new template based off of the new Forgot Password template. If it is not changed, all forgot password emails will be blank, as the forgot-password system in Magento has been completely changed. (see also Design Changes, next)

Design Changes

Templates:

New templates:
/customer/form/resetforgottenpassword.phtml

Email templates:
If the Forgot Password email template had been customized, a new template will need to be created and customized, as the forgot-password system in Magento has completely changed (see also Configuration Changes).

XML Updates:
If you have a custom "customer.xml" in your theme, it must be updated with the customer_account_resetpassword section from the base-default XML. It will not pick this up automatically. If you do not make this change, all attempts to reset a customer password will fail with a 500 server error.

Extension Issues

We're only running the AheadWorks blog, so our exposure here is rather limited.

AheadWorks Blog v1.0.22 (AW All v2.1.2)
Version numbers above 1.6 for Magento are not properly recognized, and may cause the extension to think you are running a bad version of Magento (and disable itself). You must change the version number defined in app/code/local/AW/All/Helper/Config.php .

Various Unknowns
Magento 1.6 tightens it's reins when it comes to table storage types and foreign key constraints. Any module which modifies a core table, index, or key constraint has a good chance to run into (or cause) problems (before or after) when upgrading.

Magento allows administrators to create orders that do not include an e-mail address. This makes sense as there are certainly situations where such an order would be placed without one. In cases such as these Magento will create a fictitious e-mail address for the order. By default (up through Magento 1.5.x) that e-mail address will look something like 234234252@emaildomain.com.
See the problem?

Sure, we could be thinking - "But it's just for customer orders" or "I don't check the box to send the order e-mail" or "Who makes an order in the admin?" or any number of thoughts which seem to make this a non-issue.

Unfortunately, it's a huge issue. A simple click in the wrong place and we just sent out our customers private information to a stranger. Specifically with this default setting it's entirely possible to send out a customers order information to the owners of "emaildomain.com". It makes me cringe to imagine how many they may have received.

It's all fine and good when speaking to give an example like user@domain.com or me@mycompany.com because we have a high hope of conveying that it is an example and not to be used literally. With a default configuration however, what is or isn't to be used is not so clear cut. We kind of expect that a default configuration is either acceptable or will in no way work without changing. To know what is an acceptable default value we should know that there are only four 'reserved' top-level domains. That is, domains which are not able to be resolved over public networks (and that includes e-mail). These domains are: example, invalid, localhost, and test (read more about reserved TLDs here). Which means an e-mail address of user@example.com will never (should never) go anywhere, but user@emaildomain.com will go to someone, just as user@gmail.com will - except not the person we intend or not intend as the case may be¹.

Thankfully this was noticed (how it took so long will be an untold story) and reported in this bug report. In the 1.6.x branch of Magento we don't have to worry about this little item. For those of us who haven't upgraded yet - the fix is pretty simple. In the Magento configuration go to:
Configuration->Customer Configuration->Create New Account Options
and change Default Email Domain from "emaildomain.com" to "example.com" and save.

Alternatively we can change the file: app/code/core/Mage/Customer/etc/config.xml looking for a line like:
<email_domain>emaildomain.com</email_domain>
and changing it to:
<email_domain>example.com</email_domain>
Just in case, we'll want to check the configuration in the admin of Magento to make sure it sticks.

We could set this to a different domain such as 'invalid.com' or 'test.com' or even our own company domain. With the last we need to still keep in mind what is happening - as it has consequences. The use of 'example.com' is probably best in any case.

That's all there is to it. We and our customers can now rest just a bit easier tonight.

¹ Technically, the MX record for emaildomain.com does not resolve at the time of this writing, so a misplaced e-mail still won't get anywhere. That doesn't mean it couldn't - and that's what we want to prevent.

For a little background we can read that the two issues are in Magento's field of vision with this bug 25547 which still seems open and this bug 26296 which hints that perhaps, just maybe, a fix will be in 1.6.1. A Magento forum discussion to this can be found here and as well here among other places.

Let's just fix it. The file giving grief we'll find here:
app/design/frontend/base/default/template/checkout/onepage/shipping_method/available.phtml
or your theme's version of it. If you don't have one in your theme, you'll probably want to copy the base/default one now. Towards the end of the file you'll see a javascript function includingShipping:

function includingShipping(getShippingCode)
{
	<?php if (!empty($shippingMe)): ?>
	var newPrice = shippingMe[getShippingCode];
	if (!lastPrice) {
		lastPrice = newPrice;
		quoteBaseGrandTotal += newPrice;
	}
	if (newPrice != lastPrice) {
		quoteBaseGrandTotal += (newPrice-lastPrice);
		lastPrice = newPrice;
	}

The lines we're interested in are 81, 83, and 86. There are a few ways to rewrite this code depending on your tastes. For example In line 81 we are given:

	if (!lastPrice) {

While such a conditional may pass muster with PHP, it doesn't with Javascript. This line has the chance to produce the error: lastPrice is not defined because, straightforwardly, lastPrice is not always defined. 'Not always' meaning lastPrice is conditionally defined earlier in this file at around line 50. We can work with this pretty easily either by ensuring lastPrice is always set, or re-writing the conditional which checks it to be a bit more stable like so:

	if (typeof lastPrice === 'undefined') {

Which brings us to the other occurrence of an undefined variable - quoteBaseGrandTotal. Like lastPrice, it isn't necessarily defined. So doing any math with it has a chance of ending with a bit of a sting. quoteBaseGrandTotal may be called on line 83 or 86 (or possibly both). Following the lastPrice change above, we add a check for it to see if it is defined, and from that decide how to do the math. After a bit of tape and solder we arrive with:

function includingShipping(getShippingCode)
{
	<?php if (!empty($shippingMe)): ?>
	var newPrice = shippingMe[getShippingCode];
	if (typeof lastPrice === 'undefined') {
		lastPrice = newPrice;
		if(typeof quoteBaseGrandTotal === 'undefined') {
			quoteBaseGrandTotal = newPrice;
		} else {
			quoteBaseGrandTotal += newPrice;
		}
	}
	if (newPrice != lastPrice) {
		if(typeof quoteBaseGrandTotal === 'undefined') {
			quoteBaseGrandTotal = (newPrice-lastPrice);
		} else {
			quoteBaseGrandTotal += (newPrice-lastPrice);
		}
		lastPrice = newPrice;
	}

Which should keep the math as intended and remove two possible errors from our customers shopping experience. Keep your fingers crossed for 1.6.1!

Page 1: Package Information

The first page is fairly straightforward. First, you have Name, which is the name of the package you're releasing. There's also Channel, which can vary, but should probably be the name of the company/team you're releasing it under.

Supported Releases is an option to signify the compatibility of your extension. With the release of Magento 1.6.0, I wouldn't be surprised to see a third option appear in that list, due to all the large changes that have taken place. Summary and Description should be filled with, respectively, short and detailed descriptions of the functions your extension offers, and who it is aimed at. License will store your license agreement version, with an option License URI to directly link to the terms

All the information on this page will be stored in a Metadata XML file that will be created along with your packaged extension.

Page 2: Release Information

Just as page 1 was about your extension as a whole, page 2 focuses on this specific release, since a growing extension can have several. Release Version should match the extension version you have defined in etc/config.xml, though that is not directly enforced by Magento. Notes allow you to add specific details about this release, which in most extensions should probably list new features, fixed errors, known bugs, or any other information that is especially important to a user who could be upgrading from a previous version.

Release Stability is an important field - it allows you to show which stage of release this version is! The options are:

• Development - Generally an internal build, being packaged for testing
• Alpha - A "first release" possibly, or more often the first development build where all the core features are complete
• Beta - The extension is nearing full release, and you may want to put out a more publicly available Beta version for experienced users to test
• Stable - Your extension is ready for the public, with minimal bugs or errors known.

Page 3: Authors

Page 3 is very simple - you just list out all the developers/authors for this extension! You can add as many authors here as you would like.

Page 4: Dependencies

Dependencies are a step that many extensions will probably need to pay very little attention to, but is also an incredibly important detail to some! If you don't properly define what your extension depends on, users without proper requirements may try and use it to no avail, resulting in bad reviews or a very bad user experience.

Minimum and Maximum PHP Versions are fairly direct. The minimum should be the earliest version that your most modern PHP language feature was introduced - in most cases, a rough milestone version like 5.1.0 or 5.2.0 is good enough if you do not know exactly when the feature was introduced, but you should try your best to find the exact release. Maximum version is usually of much less concern, as you only need to worry about it if a feature you use is marked for deprecation or has been removed. Setting this to a future release of PHP is often an acceptable option.

PackagesExtensions are where you define the names and versions of PHP extensions or add-ons that you make use of in your Magento Extension. A common example could be the GD Library, used for generating images - it is not a guaranteed on every user's PHP installation. Make sure to define any external PHP libraries you make use of here, so users know what is required by your extension to use it.

Page 5: Contents

Finally! The meat and bones of the extension packaging process - defining exactly what gets packaged! This page can be a bit confusing at first. The Target field is the section of Magento's directory structure you're pulling the files from. Here is a direct definition of what each of those options point to:

• Magento Local module file: ./app/code/local
• Magento Community module file: ./app/code/community
• Magento Core team module file: ./app/code/core
• Magento User Interface (layouts, templates): ./app/design
• Magento Global Configuration: ./app/etc
• Magento PHP Library file: ./lib
• Magento Locale language file: ./app/locale
• Magento Media library: ./media
• Magento Theme Skin (Images, CSS, JS): ./skin
• Magento Other web accessible file: .
• Magento PHPUnit test: ./tests
• Magento other: .

The Path should be relative to the Target you selected, with the Type noting whether you've pointed to a direct file or a directory. Most extensions will have at least 4 entries here - the path to the code, the path to the templates, the path to the module config file, and the path to the layout config file. Of course, some modules may have more or even less - that all depends on what you're doing!

The Include and Ignore fields allow you to limit what gets packaged into the extension from a recursive directory input. It takes wildcard inputs looking for filetypes or otherwise (for example, if you kept a log in an included file - you could add "*.log" to the Ignore section), and uses the "#" character as a delimiter. If your extension properly respects the Magento design method of separating logic and output, these will most likely be unnecessary for almost all extensions.

Saving and Accessing Your Extension

At this point, you have defined everything Magento needs to know about your extension. Clicking "Save As" will let you name your extension package within Magento (we'll return to this). Save Data and Create Package will then commit your newest version to the record, and create a packaged file for you. You can find that file in your var/connect folder, named YourExtension_Version.tgz. You will also find the file YourExtension.xml, which has all the metadata you inserted in step 1. From here, you can use these files to import your extension easily through Magento Connect to any installation of Magento!

Making Changes or a New Version (Page 6)

So, you've packaged and released your extension. But like any piece of software, it will need changes! Fixes and new features are commonplace. Luckily, once you've created your extension, you don't have to go through the hassle of inserting all the same information over and over again. Once you save it, you can easily load it from Page 6: Load Local Package any time you want. Clicking one of your packages on this screen will populate the other pages with all the information you previously saved, meaning you only have to make minor changes - version, notes, and maybe a folder change or two. It makes rolling out new releases quick and simple.

Hopefully this guide helps you get your site's extensions ready to roll out. If you have any questions I didn't address here, post them in the comments and we can try and help you out!

Not a few people have reported in various forms that uploading media files in the admin is broken. At least the appearance of being broken. If you find that when you upload a file or create a folder in the WYSIWYG editor in the admin and then it seems to disappear - then maybe we've walked the same path. What' makes this issue fun is that you don't see an error message. Folders and files just seem to vanish. We all know disappearing resources can make even the Doctor concerned, so let's get to the bottom of it.

You're not going to get a reminder to check your folder permissions, you've already done that (for the love of everything good please have tried that already). If you hit the forums, you've probably already read through this thread, or this thread mentioning two fixes, or any number of other discussions which talk about not being able to create folders. Again, the key to to this issue is that it appears as if they are not created, while browsing the file system shows that the uploaded files and directories created are indeed there. This might fool us into thinking that Magento is having a problem reading the file system.

Magento can read the file system just fine. What it can't read is a table in the database. Mostly because it hasn't created the table. Welcome to your bug.

The second thread above actually does note a fix by creating the table, however there is another way that will put things right without guessing at what code to write. The issue (in this case) is that Magento is looking for a table or two it didn't create (yet). They are 'core_file_storage' and 'core_directory_storage'. This problem is mentioned here directly. There is an error message stating one or both of those tables do not exist. Of course you won't find that thread so easily if you don't see an error message.

The problem is that the Magento module responsible (it's called core) for this storage functionality doesn't create the tables as we are taught in the 'Magento Way' with the _mysql setup files. Magento only creates the tables only if the database is set to be used as the media storage. That kind of decision makes some sense, but it's only good if it's followed through.

Check out app/code/core/Mage/Core/etc/config.xml and you'll notice in global/models/core_mysql/entities two tables file_storage and directory_storage noted. Then check that version number and jump over to app/code/core/Mage/Core/sql/core_setup/. Find your version number and read the setup file. See any table creation that mentions either table? Do any of the setup files? If you're using any of the 1.5 releases chances are you are shaking your head.

There is a table creation in app/code/core/Mage/Core/Model/Mysql4/File/Storage/Database.php - however that is only run when the storage setting is the database, not when Magento is setup. And there is half of what we are looking for.

So if your folders and uploaded files seem to disappear or not show up in the Magento WYSIWYG editor, but actually do exist on the filesystem, then the following will probably work for you:

• Go to System->Configuration->Advanced/System->Storage Configuration Media;
• Change Media Storage to 'Database';
• Click Syncronize;
• Wait for "syncronizing message" to stop showing, refill your mug;
• Change Media Storage back to 'File System';
• Fini
• Bonus: There's no need to save the configuration as nothing has changed!
• Double Bonus: Feel free to truncate the two tables 'core_directory_storage' and 'core_files_storage' from you database if you like. They aren't actually used for storage if you are using the file system.

How does Magento fix it in 1.6? They dropped the functionality.

This brings up two entirely different issues. One, if the storage system is set to use the File System, why is the Magento checking the database for the files or directories to begin with? And two, why isn't an error being thrown when it can't locate a resource? The former is an enigma, the latter the subject of another article...