The key idea in what some call the 'right to be forgotten' is the recognition that personal information should not be kept for longer than needed. It also recognises that an individual has the right to request that their personal information is erased. The holder must then either erase it or provide a good reason why this won't happen.
What counts as a good enough reason? One example is that the information is needed to supply a service that a customer still wants or needs. It's not possible to provide a banking service for a person, but also to erase all their personal information from the bank's systems, for example. Another is that the holder is required by laws or regulations to keep personal information for a set period before it's erased. In fact this is the case for banks - there are strict rules about the records banks must keep.
For example, banks are required to hold financial records to help fight crimes such as fraud, money laundering, or terrorism. In relation to banks it's misleading to think of a simple 'right to be forgotten'. You have a right to request personal information to be erased, and for this to be carried out, or to be given a satisfactory reason why it can't be.