Unix uniq command

• sohguanh

  9 Feb 06, 09:54

  This forum is for IT related problems?
  
  I have a problem. I need to use Unix uniq command to remove duplicating lines but with an extra twist.
  
  Example
  1 apple a
  2 apple b
  3 orange c
  4 orange d
  ....
  
  I wanna the command to consider the middle field only (i.e apple, orange) to do removing of duplicates. But it seem uniq is treating the whole line to do comparison when remove duplicates. Any idea how?
  
  Dun ask me use M$ Excel as the command is supposed to be in a Unix script to be executed on commandline by data centre operators in a Unix environment.
  
  TIA
  
• ndmmxiaomayi

  9 Feb 06, 10:27

  I never use Unix uniq before....
  
  You wait for launtpc to come in. He is programming expert.
• launtpc

  9 Feb 06, 15:03

  I can't help him... I don't know anything in Unix...
• FX_82

  10 Feb 06, 15:17

  Originally posted by sohguanh:

  This forum is for IT related problems?
  
  I have a problem. I need to use Unix uniq command to remove duplicating lines but with an extra twist.
  
  Example
  1 apple a
  2 apple b
  3 orange c
  4 orange d
  ....
  
  I wanna the command to consider the middle field only (i.e apple, orange) to do removing of duplicates. But it seem uniq is treating the whole line to do comparison when remove duplicates. Any idea how?
  
  Dun ask me use M$ Excel as the command is supposed to be in a Unix script to be executed on commandline by data centre operators in a Unix environment.
  
  TIA
  

  http://www.sikh-history.com/computers/unix/commands.html#uniqcommand
  try this syntax bah.hope can help u
• KumSioJui

  27 Feb 06, 10:00

  Originally posted by sohguanh:

  This forum is for IT related problems?
  
  I have a problem. I need to use Unix uniq command to remove duplicating lines but with an extra twist.
  
  Example
  1 apple a
  2 apple b
  3 orange c
  4 orange d
  ....
  
  I wanna the command to consider the middle field only (i.e apple, orange) to do removing of duplicates. But it seem uniq is treating the whole line to do comparison when remove duplicates. Any idea how?
  
  Dun ask me use M$ Excel as the command is supposed to be in a Unix script to be executed on commandline by data centre operators in a Unix environment.
  
  TIA
  

  uniq command can do only limited. I normally use awk. awk is more powerful.
  
  #cat /tmp/xxx
  1 apple a
  2 apple b
  3 orange c
  4 orange d
  #
  #
  
  #cat /tmp/xxx | awk '$2 =="apple" { print }'
  1 apple a
  2 apple b
  #
  
  I have a file called xxx under /tmp
  
  in the awk statement $2 refers to the second middle field, so the whole command cat and awk statement what is does is to display contents of the file and pipe it to awk which does filtering searches for the string called "apple" and print any occurences it finds.
  
  hope it helps
  
  another example to
  
  #cat /tmp/xxx | awk '$3 =="d" {print }'
  4 orange d
  #