2003–2004

In July, & was released—Hamasaki’s first single in nine months. Like the single H, & featured three a-sides, but & also included a b-side, theme of a-nation '03. This was Hamasaki's last single to sell over 500,000 units on the Oricon Charts. The follow up single, Forgiveness would continue Hamasaki's string of number one singles, but underperformed sales-wise in contrast to her previous efforts, barely selling 220,000 copies in contrast to the sales of &, which netted sales of over 570,000. After the releases of the singles Forgiveness in August and No Way to Say in November, she released her first mini-album, Memorial Address, which only contained three new songs. Memorial address, Hamasaki's first album to be available in CD only or CD+DVD formats, also made her the only female solo artist to have a million-selling mini-album.

Main article: My Story (Ayumi Hamasaki album)
In spring 2004, Hamasaki released Moments in both a CD only and CD+DVD combination, something which she continues to do now. It was featured in her last Kose Visee commercial before her contract with the cosmetics company ended. In July that year, internal strife in her recording company, Avex Trax, occurred. A feud between Max Matsuura and Avex's then-president Tom Yoda, regarding Avex business practices, caused chaos and also sparked rumors that Hamasaki and other popular artists in Avex Trax including Hiro, Every Little Thing, Hitomi, and Do As Infinity might leave the record company and follow Max Matsuura. As a result, the stock prices for Avex Trax began to plunge; when Hamasaki sided with Matsuura, Yoda ended up resigning, effectively ending the feud.

In December 2004, she released her sixth studio album, My Story, which was certified million-status, selling over 574,000 copies in the first week, and netting sales of over 1,140,000 copies, making it the second highest-selling album released by a female artist in the 2005 Oricon year