You can put a partially filled Memory into a Diary, but you can't add more Experiences to that Memory after you do so.  

When you put a Memory into a Diary your character functionally forgets it.  That Memory slot is now free for all new Memories, the Memory that was there is now a Diary entry.

You should only be putting Memories into a Diary when you are out of Memory slots.  Don't casually add three Memories to a Diary, that's just not the way the game is supposed to work mechanically.  

Does this help?