Hello All,
I have a scenario in which i am getting records from users like -:
Userid,Name,Territory,country,StageID
So the records are coming in a stream which i am getting from kafka.
The scenario is like 1st record i get is like-:
12,Sachin,Delhi,Delhi,1
again if the record is updated then it is pushed again in kafka and i get-:
12,Sachin,Delhi,Delhi,2
So what i need to do is i need to count the records in the basis of stages and if a particular record moves from 1 stage to other then that stage should reflect 1 less record and other stage would be reflecting 1 more record.
Kindly help on this.
Till now i have seen spark structured streaming and which seems to solve the above problem. So is there anything else which can solve the above scenario.
Regards,
Sachin