How to eliminate null dataframe in spark streaming before saving to table or upload to cloud.
if we do some transformation like val aname = df.filter(col(“name”).isNotNull)
here it will filter and then create a new streaming df “aname” now if we want to check record count for this df and on that basis want to save data or upload to s3 bucket then I am not finding any option like static df.
I am not able to use count() or isEmpty or even converting to dataset or rdd not working